Publications

Refine Results

(Filters Applied) Clear All

Tagged As

Development of a new inanimate class for the WSR-88D hydrometeor classification algorithm

August 31, 2017
  |
Conference Paper
Author:
James M. Kurdzo
…
Published in:
38th Conf. on Radar Meteorology, 27 August-1 September 2017.
Topic:
algorithms
R&D area:
R&D group:

Summary

The current implementation of the Hydrometeor Classification Algorithm (HCA) on the WSR-88D network contains two non-hydrometeor-based classes: ground clutter/anomalous propagation and biologicals. A number of commonly observed non-hydrometeor-based phenomena do not fall into either of these two HCA categories, but often are misclassified as ground clutter, biologicals, unknown, or worse yet, weather hydrometeors. Some of these phenomena include chaff, sea clutter, combustion debris and smoke, and radio frequency interference. In order to address this discrepancy, a new class (nominally named "inanimate") is being developed that encompasses many of these targets. Using this class, a distinction between non-biological and biological non-hydrometeor targets can be made and potentially separated into sub-classes for more direct identification. A discussion regarding the fuzzy logic membership functions, optimization of membership weights, and class restrictions is presented, with a focus on observations of highly stochastic differential phase estimates in all of the aforementioned targets. Recent attempts to separate the results into sub-classes using a support vector machine are presented, and examples of each target type are detailed. Details concerning eventual implementation into the WSR-88D radar product generator are addressed.
READ LESS

Summary

The current implementation of the Hydrometeor Classification Algorithm (HCA) on the WSR-88D network contains two non-hydrometeor-based classes: ground clutter/anomalous propagation and biologicals. A number of commonly observed non-hydrometeor-based phenomena do not fall into either of these two HCA categories, but often are misclassified as ground clutter, biologicals, unknown, or worse...

READ MORE
Development of a new inanimate class for the WSR-88D hydrometeor classification algorithm

A new radio frequency interference filter for weather radars

July 1, 2017
  |
Journal Article
Author:
John Y. Cho
Published in:
J. Atmos. Ocean. Technol., Vol. 34, No. 7, 1 July 2017, pp. 1393-1406.
Topic:
algorithms
R&D area:
R&D group:

Summary

A new radio frequency interference (RFI) filter algorithm for weather radars is proposed in the two-dimensional (2D) range-time/sample-time domain. Its operation in 2D space allows RFI detection at lower interference-to-noise or interference-to-signal ratios compared to filters working only in the sample-time domain while maintaining very low false alarm rates. Simulations and real weather radar data with RFI are used to perform algorithm comparisons. Results are consistent with theoretical considerations and show the 2D RFI filter to be a promising addition to the signal processing arsenal against interference with weather radars. Increased computational burden is the only drawback relative to filters currently used by operational systems.
READ LESS

Summary

A new radio frequency interference (RFI) filter algorithm for weather radars is proposed in the two-dimensional (2D) range-time/sample-time domain. Its operation in 2D space allows RFI detection at lower interference-to-noise or interference-to-signal ratios compared to filters working only in the sample-time domain while maintaining very low false alarm rates. Simulations...

READ MORE
A new radio frequency interference filter for weather radars

Automated provenance analytics: a regular grammar based approach with applications in security

June 22, 2017
  |
Conference Paper
Author:
Mark J. Lemay
…
Published in:
9th Intl. Workshop on Theory and Practice of Provenance, TaPP, 22-23 June 2017.
Topic:
algorithms
R&D area:
R&D group:

Summary

Provenance collection techniques have been carefully studied in the literature, and there are now several systems to automatically capture provenance data. However, the analysis of provenance data is often left "as an exercise for the reader". The provenance community needs tools that allow users to quickly sort through large volumes of provenance data and identify records that require further investigation. By detecting anomalies in provenance data that deviate from established patterns, we hope to actively thwart security threats. In this paper, we discuss issues with current graph analysis techniques as applied to data provenance, particularly Frequent Subgraph Mining (FSM). Then we introduce Directed Acyclic Graph regular grammars (DAGr) as a model for provenance data and show how they can detect anomalies. These DAGr provide an expressive characterization of DAGs, and by using regular grammars as a formalism, we can apply results from formal language theory to learn the difference between "good" and "bad" provenance. We propose a restricted subclass of DAGr called deterministic Directed Acyclic Graph automata (dDAGa) that guarantees parsing in linear time. Finally, we propose a learning algorithm for dDAGa, inspired by Minimum Description Length for Grammar Induction.
READ LESS

Summary

Provenance collection techniques have been carefully studied in the literature, and there are now several systems to automatically capture provenance data. However, the analysis of provenance data is often left "as an exercise for the reader". The provenance community needs tools that allow users to quickly sort through large volumes...

READ MORE
Automated provenance analytics: a regular grammar based approach with applications in security

Fabrication security and trust of domain-specific ASIC processors

May 1, 2017
  |
Conference Paper
Author:
Mankuan Michael Vai
…
Published in:
IEEE Intl. Symp. on Hardware Oriented Security and Trust, HOST, 1-5 May 2017.
Topic:
algorithms
R&D area:
R&D group:

Summary

Application specific integrated circuits (ASICs) are commonly used to implement high-performance signal-processing systems for high-volume applications, but their high development costs and inflexible nature make ASICs inappropriate for algorithm development and low-volume DoD applications. In addition, the intellectual property (IP) embedded in the ASIC is at risk when fabricated in an untrusted foundry. Lincoln Laboratory has developed a flexible signal-processing architecture to implement a wide range of algorithms within one application domain, for example radar signal processing. In this design methodology, common signal processing kernels such as digital filters, fast Fourier transforms (FFTs), and matrix transformations are implemented as optimized modules, which are interconnected by a programmable wiring fabric that is similar to the interconnect in a field programmable gate array (FPGA). One or more programmable microcontrollers are also embedded in the fabric to sequence the operations. This design methodology, which has been termed a coarse-grained FPGA, has been shown to achieve a near ASIC level of performance. In addition, since the signal processing algorithms are expressed in firmware that is loaded at runtime, the important application details are protected from an unscrupulous foundry.
READ LESS

Summary

Application specific integrated circuits (ASICs) are commonly used to implement high-performance signal-processing systems for high-volume applications, but their high development costs and inflexible nature make ASICs inappropriate for algorithm development and low-volume DoD applications. In addition, the intellectual property (IP) embedded in the ASIC is at risk when fabricated in...

READ MORE
Fabrication security and trust of domain-specific ASIC processors

Benchmarking the Graphulo processing framework

September 13, 2016
  |
Conference Paper
Author:
Vijay N. Gadepally
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
high performance computing
R&D area:
R&D group:

Summary

Graph algorithms have wide applicability to a variety of domains and are often used on massive datasets. Recent standardization efforts such as the GraphBLAS are designed to specify a set of key computational kernels that hardware and software developers can adhere to. Graphulo is a processing framework that enables GraphBLAS kernels in the Apache Accumulo database. In our previous work, we have demonstrated a core Graphulo operation that performs large scale multiplication operations of database tables called TableMult. In this article, we present results of scaling the Graphulo engine to larger problems and scalablity when using greater number of resources. Specifically, we present the results of two experiments that demonstrate Graphulo scaling performance as linear with the number of available resources. The first experiment demonstrates cluster processing rates through Graphulo's TableMult operator on two large graphs, scaled between 2^17 and 2^19 vertices. The second experiment uses TableMult to extract a random set of rows from a large graph (2^19 nodes) to simulate a cued graph analytic. These benchmarking results are of relevance to Graphulo users who wish to apply Graphulo to their graph problems.
READ LESS

Summary

Graph algorithms have wide applicability to a variety of domains and are often used on massive datasets. Recent standardization efforts such as the GraphBLAS are designed to specify a set of key computational kernels that hardware and software developers can adhere to. Graphulo is a processing framework that enables GraphBLAS...

READ MORE
Benchmarking the Graphulo processing framework

From NoSQL Accumulo to NewSQL Graphulo: design and utility of graph algorithms inside a BigTable database

September 13, 2016
  |
Conference Paper
Author:
Dylan D. Hutchison
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
algorithms
R&D area:
R&D group:

Summary

Google BigTable's scale-out design for distributed key-value storage inspired a generation of NoSQL databases. Recently the NewSQL paradigm emerged in response to analytic workloads that demand distributed computation local to data storage. Many such analytics take the form of graph algorithms, a trend that motivated the GraphBLAS initiative to standardize a set of matrix math kernels for building graph algorithms. In this article we show how it is possible to implement the GraphBLAS kernels in a BigTable database by presenting the design of Graphulo, a library for executing graph algorithms inside the Apache Accumulo database. We detail the Graphulo implementation of two graph algorithms and conduct experiments comparing their performance to two main-memory matrix math systems. Our results shed insight into the conditions that determine when executing a graph algorithm is faster inside a database versus an external system—in short, that memory requirements and relative I/O are critical factors.
READ LESS

Summary

Google BigTable's scale-out design for distributed key-value storage inspired a generation of NoSQL databases. Recently the NewSQL paradigm emerged in response to analytic workloads that demand distributed computation local to data storage. Many such analytics take the form of graph algorithms, a trend that motivated the GraphBLAS initiative to standardize...

READ MORE
From NoSQL Accumulo to NewSQL Graphulo: design and utility of graph algorithms inside a BigTable database

Enforced sparse non-negative matrix factorization

May 23, 2016
  |
Conference Paper
Author:
Brendan E. Gavin
…
Published in:
30th IEEE Int. Parallel and Distributed Processing Symp., IPDPS 2016, 23-27 May 2016.
Topic:
supercomputing
R&D area:
R&D group:

Summary

Non-negative matrix factorization (NMF) is a dimensionality reduction algorithm for data that can be represented as an undirected bipartite graph. It has become a common method for generating topic models of text data because it is known to produce good results, despite its relative simplicity of implementation and ease of computation. One challenge with applying the NMF to large datasets is that intermediate matrix products often become dense, thus stressing the memory and compute elements of the underlying system. In this article, we investigate a simple but powerful modification of the alternating least squares method of determining the NMF of a sparse matrix that enforces the generation of sparse intermediate and output matrices. This method enables the application of NMF to large datasets through improved memory and compute performance. Further, we demonstrate, empirically, that this method of enforcing sparsity in the NMF either preserves or improves both the accuracy of the resulting topic model and the convergence rate of the underlying algorithm.
READ LESS

Summary

Non-negative matrix factorization (NMF) is a dimensionality reduction algorithm for data that can be represented as an undirected bipartite graph. It has become a common method for generating topic models of text data because it is known to produce good results, despite its relative simplicity of implementation and ease of...

READ MORE
Enforced sparse non-negative matrix factorization

Airspace flow rate forecast algorithms, validation, and implementation

December 15, 2015
  |
Project Report
Author:
Michael P. Matthews
…
Published in:
MIT Lincoln Laboratory Report ATC-428
Topic:
artificial intelligence
R&D area:
R&D group:

Summary

This report summarizes work performed by MIT Lincoln Laboratory during the period 1 February 2015 - 30 November 2015 focused on developing and improving algorithms to estimate the impact of convective weather on air traffic flows. The core motivation for the work is the need to improve strategic traffic flow management decision-making in the National Airspace System. The algorithms developed as part of this work translate multiple weather forecast products into a discrete airspace impact metric called permeability.
READ LESS

Summary

This report summarizes work performed by MIT Lincoln Laboratory during the period 1 February 2015 - 30 November 2015 focused on developing and improving algorithms to estimate the impact of convective weather on air traffic flows. The core motivation for the work is the need to improve strategic traffic flow...

READ MORE
Airspace flow rate forecast algorithms, validation, and implementation

Enhanced signal processing algorithms for the ASR-9 Weather Systems Processor

October 1, 2015
  |
Journal Article
Author:
John Y. Cho
Published in:
J. Atmos. Ocean. Technol., Vol. 32, No. 10, October 2015, pp. 1847-59.
Topic:
algorithms
R&D area:
R&D group:

Summary

New signal processing algorithms for the Airport Surveillance Radar-9 (ASR-9) Weather Systems Processor (WSP) are introduced. The Moving Clutter Spectral Processing for Uneven-Sampled Data with Dealiasing (MCSPUDD) algorithm suite removes isolated moving clutter targets and corrects aliased velocity values on a per-range-gate basis. The spectral differencing technique is applied to the low- and high-beam data to produce a dual-beam velocity estimate that is more accurate than the current autocorrelation-lag-1-based approach. Comparisons with Terminal Doppler Weather Radar (TDWR) data show that estimate errors are reduced by 8%, 15%, and 15% for the low-, high-, and dual-beam velocities, respectively.
READ LESS

Summary

New signal processing algorithms for the Airport Surveillance Radar-9 (ASR-9) Weather Systems Processor (WSP) are introduced. The Moving Clutter Spectral Processing for Uneven-Sampled Data with Dealiasing (MCSPUDD) algorithm suite removes isolated moving clutter targets and corrects aliased velocity values on a per-range-gate basis. The spectral differencing technique is applied to...

READ MORE
Enhanced signal processing algorithms for the ASR-9 Weather Systems Processor

Velocity estimation improvements for the ASR-9 Weather Systems Processor

February 2, 2014
  |
Conference Paper
Author:
James M. Kurdzo
…
Published in:
American Meteorological Society Annual Meeting, 2-6 February 2014.
Topic:
algorithms
R&D area:
R&D group:

Summary

The Airport Surveillance Radar (ASR-9) is a rapid-scanning terminal aircraft detection system deployed at airports around the United States. To provide cost-effective wind shear detection capability at medium-density airports, the Weather Systems Processor (WSP) was developed and added on to the ASR-9 at 35 sites. The WSP on the ASR-9 is capable of utilizing dual fan-beam estimates of reflectivity and velocity in order to detect low-level features such as gust fronts, wind shear, and microbursts, which would normally be best detectable by a low-scanning pencil beam radar. An upgrade to the ASR-9 WSP, which is currently ongoing, allows for additional computational complexity in the front-end digital signal processing algorithms compared to previous iterations of the system. This paper will explore ideas for improving velocity estimates, including low-level dual beam weight estimation, de-aliasing, and noise reduction. A discussion of the unique challenges afforded by the ASR-9's block-stagger pulse repetition time is presented, along with thoughts on how to overcome limitations which arise from rapid-scanning and the inherent lack of pulses available for coherent averaging.
READ LESS

Summary

The Airport Surveillance Radar (ASR-9) is a rapid-scanning terminal aircraft detection system deployed at airports around the United States. To provide cost-effective wind shear detection capability at medium-density airports, the Weather Systems Processor (WSP) was developed and added on to the ASR-9 at 35 sites. The WSP on the ASR-9...

READ MORE
Velocity estimation improvements for the ASR-9 Weather Systems Processor
11-20 of 88