Project Documents

  1. TD01 Algorithm Theoretical Basis Document

    This document provides a basic algorithm description for aerosol classification from optical data using the benefits of artificial neural networks: capability to learn nonlinear complex relations, to tolerate quite noisy signals and to classify in real time after a proper training process. The method will be developed in two steps: the first step consists in the classification of a large set of data both from simulations and observations for training and in a second step we will do fine adjustments in order to increase the number of best classified aerosol types. The goal is to use as many optical parameters, which are measured by a ground based multiwavelength depolarization Raman lidar, as possible.

    Full document available within the restricted area

  2. TD02 Report on the synthetic database content

    This document describes the methodology and the tools used to compute synthetic optical properties of aerosols, as well as the content of the database. The purpose is to obtain a relevant number of training cases for the artificial neural network, considering both pure and mixed aerosol types. Aerosols were simulated starting from the microphysical properties of basic components, internally mixed in various proportions. Validation of the values computed was done by comparison to measurements published in the literature.

    Full document available within the restricted area

  3. TD03 List of input parameters considered and ranges

    This document describes the variables chosen as input parameters in the Artificial Neural Network(ANN) algorithm for aerosol typing.
    Ranges of the values computed for pure aerosol classes have been compared to measurements published in the literature. Six classes of pure aerosol have been considered as were previously defined in NATALI-TD01 -01_v01.

    Full document available within the restricted area

  4. TD04 Table list of the software technical requirements

    This document provides the list of principal software requirements for aerosol classification from optical data derived from lidar measurements. Diverse Artificial neural networks (ANN) algorithms and designs will be used in order to best recognize several pure and mixt aerosols types. At least two ANN configurations will be used depending on the number of parameters considered. Data from simulations and observations will be used to train, test and adjust the ANN performances.

    Full document available within the restricted area

  5. TD05 List of cases selected and confirmed aerosol type

    This document explains the methodology used to select observational data which are proper to be used in the process of training and testing the artificial neural network. It also presents the statistics of the selected cases, results obtained in comparison to synthetic data, and proposes a strategy to optimize the performance and the use of the aerosol typing algorithm.

    Full document available within the restricted area

  6. TD06 Schematics of the neural network

    This document shows the basic schematics of the artificial neural networks which will be used to further classify the aerosols type. Few types of neural networks are selected depending on the number of aerosols types to be retrieved and performances of the training and classification processes. Several comparative results on ANN performances using synthetic data are also presented.

    Full document available within the restricted area

  7. TD07 Report on software design

    This document presents the design of the software which implements the ANN algorithms for aerosol typing.

    Full document available within the restricted area

  8. TD08 Report on lidar, in-situ and integrated column data collected

    This document presents the report on lidar, in-situ and integrated column data collected.

    Full document available within the restricted area

Results

Synthetic data

The synthetic database is generated by simulating the optical properties of various aerosol types based on available information on the microphysics. The algorithm combines the GADS database (Global Aerosol DataSet) to OPAC model (Optical Properties of Aerosol and Clouds) and T-Matrix code in order to compute, in an iterative way, the intensive optical properties of each aerosol type. The chemical composition of each aerosol type varies in certain limits, in order to mimic as much as possible the large variety of particles present in the atmosphere.

Each aerosol type (pure) is built as an internal mixture of typical components which do not interact physically or chemically, having different mixing ratios. Basic components are picked up from OPAC (Hess et al 1998): water soluble, insoluble, soot, mineral (nucleation, accumulation, coarse), sulfates, sea salt (accumulation, coarse). GADS database is used for the microphysical properties of each component (Koepke et al., 1997). In order to include the non-sphericity of particles, T-matrix code is used to calculate aerosol’s optical properties (Waterman, 1965). Aerosols are considered spheroids with different axis ratios. These values were picked up from the literature (Munoz et al., 2001; Dubovik et al., 2006).

Schedule

Artificial Neural Networks

An ANN represents a mathematical projection of the human neural network. It is based on neurons, axons and synapses, the information being propagated as a neural influx. The analysis process takes part during the propagation of information through the synapses, where it is adjusted to properly interpret the given data. The ANN unit consists of neurons at each layer’s base, which include input, output and hidden layers. The outputs of the first layer become the input to the next layer (Braspenning et al., 1995).

Although neural networks can deal with complex and interconnected input vectors, which cannot be well separated using other classification methods, the efficiency of the classification depends strongly on the complexity of the input data, as well as on the training process. The input data must be constrained to a pattern and the ANN should learn to recognize this pattern. In the learning process, some weights are established for each synapse.

Forty-eight ANN structures were tested, out of which three were selected to make the classification, based on their performances: Jordan/Elman with 6 and 8 hidden layers respectively, and Generalized feedforward with 6 hidden layers. Supervised training with Momentum or Conjugate gradient learning rule was applied. Two levels of resolution have been considered in terms of ANN outputs, specifically to provide in the same time a rough classification out of six aerosol types (min. 70% pure and 30% residuals), and a detailed classification out of fourteen aerosol types (six 90% pure, six mixtures of two types, two mixtures of three). High resolution typing is proper for advanced lidar systems which provide highly accurate optical data (less than 10% uncertainty). Low resolution typing can still be performed on optical data with an uncertainty less than 20%.

NATALI software

The software is developed in LabVIEW and is organized in two modules:

  • Input module – to prepare the lidar data in the specific format of the ANN
  • Typing module – to retrieve the aerosol type using a combination of ANN structures and “voting” procedures

The input module accommodates optical profiles in EARLINET standard NetCDF format, for multi-wavelength Raman lidar systems (3b + 2a) with or without depolarization. It finds the layer boundaries and calculates the mean values of the intensive optical parameters, and the associated uncertainties. Multiple datasets are generated taking into consideration all possible values between the error bars, and converted to the ANN format.

The typing module is designed in a two level approach. The first level takes into consideration high resolution typing, i.e. 14 aerosol (6 pure and 8 mixed aerosol types). The second level takes into consideration low resolution typing (6 basic aerosol types). The definition of types at each of the resolutions is described in TD05 (link la internal). Three ANNs produce the classification independently, for high and low resolution typing respectively. A recommendation is made by the software regarding the final aerorol type, based on a voting procedure.

The software allows sequential retrieval of aerosol type for all layers and visualization of the profile retrieved. Analysis of multiple files is also implemented.

Schedule
Literature
  1. Nemuc A. et al., Assessment of aerosol's mass concentrations from measured linear particle depolarization ratio (vertically resolved) and simulations, Atmos. Meas. Tech., 6, 3243-3255, 2013
  2. Nicolae D. et al., Characterization of fresh and aged biomass burning events using multiwavelength Raman lidar and mass spectrometry, J. Geophys. Res. Atmos., 118, doi:10.1002/jgrd.50324, 2013
  3. Vasilescu J. et al., Analysis of seawater pollution using neural networks and channels relationship algorithms, Rom. Journ. Phys., 56 (3–4), 530–539, 2011

    4. More references

  4. Adler G. et al., Chemical, physical, and optical evolution of biomass burning aerosols: a case study, Atmos. Chem. Phys., 11, 1491-1503, 2011
  5. Aires F., Neural network uncertainty assessment using Bayesian statistics with application to remote sensing: 1. Network weights, J. Geophys. Res., 109, D10303, 2004
  6. Alados-Arboledas, L. et al., Optical and microphysical properties of fresh biomass burning aerosol retrieved by Raman lidar, and star-and sun photometry, Geophys. Res. Lett., 38, L01807, doi:10.1029/2010GL045999, 2011
  7. Ali S. E. et al., Enhancement of OMI aerosol optical depth data assimilation using artificial neural network, Neural Comput & Applic 23:2267–2279, 2013
  8. Almhdi K.M. et al., Classification with Artificial Neural Networks and Support Vector Machines: Application to oil fluorescence spectra, EARSeL eProceedings 6, 2, 115-129, 2007
  9. Ansmann A., et al., European pollution outbreaks during ACE 2: Optical particle properties inferred from multiwavelength lidar and star-Sun photometry, Journal of Geophysical Research, Vol. 107, No. D15, 4259, 10.1029/2001JD001109, 2002
  10. Ansmann A., et al., The 16 April 2010 major volcanic ash plume over central Europe: EARLINET lidar and AERONET photometer observations at Leipzig and Munich, Germany, , 37, L13810, doi:10.1029/2010GL043809, 2010
  11. Bishop C. M., Neural Networks for Pattern Recognition, Clarendon, Oxford, U. K., 1995
  12. Braspenning P.J. et al., Artificial Neural Network - An Introduction to ANN Theory and Practice, Springer, ISBN 3-540-59488-4, 11-18, 1995
  13. Burton S. P. et al., Aerosol classification using airborne high spectral resolution lidar measurements – methodology and examples, Atmos. Meas. Tech., 5, 73–98, doi:10.5194/amt-5-73-2012, 2012
  14. Cattrall C. et al., Variability of aerosol and spectral lidar and backscatter and extinction ra- tios of key aerosol types derived from selected Aerosol Robotic Network locations, J. Geophys. Res.-Atmos., 110, D10S11, doi:10.1029/2004jd005124, 2005
  15. DeCarlo, P. F. et al., 2006: Field-deployable, high-resolution, time-of-flight aerosol mass spectrometer, Anal. Chem., 78, 8281–8289.
  16. Draxler R. R. and Rolph G. D., HYSPLIT (HYbrid Single-Particle Lagrangian Integrated Trajectory) Model access via NOAA ARL READY Website, available at: http://ready.arl.noaa.gov/HYSPLIT.php, NOAA Air Resources Laboratory, Silver Spring, MD., 2012
  17. Dubovik, O., B. Holben, T.F. Eck, A. Smirnov, Y.J. Kaufman, M.D. King, D. Tanre, I. Slutsker, Variability of Absorption and optical Properties of Key Aerosol Types Observed in Worldwide Locations, J. Atm. Sciences, Vol 59, 590 – 608, 2002
  18. Dubovik, O., Sinyak, A., Lapyonok, T., Holben, B. N., Mishchenko, M., Yang, P., Eck, T. F., Volten, H., Mu˜noz, O., Veihelmann, B., van der Zande, W. J., Leon, J.-F., Sorokin, M., and Slutsker, I., Application of spheroid models to account for aerosol particle nonsphericity in remote sensing of desert dust, J. Geophys. Res., 111, D11208, doi:10.1029/2005JD006619, 2006.
  19. Freudenthaler V. et al., Depolarization ratio profiling at several wavelengths in pure Saharan dust during SAMUM 2006., Tellus, B61, 165–179, doi:10.1111/j.1600-0889.2008.00396.x, 2009
  20. Gasteiger et al., Modelling lidar-relevant optical properties of complex mineral dust aerosols, Tellus (2011), 63B, 725–741, DOI: 10.1111/j.1600-0889.2011.00559.x
  21. Groß S. et al., Characterization of saharan dust, marine aerosols and mixtures of biomass burning aerosols and dust by means of multi-wavelength depolarization- and Raman-measurements during SAMUM-2, Tellus, B63, 706–724, doi:10.1111/j.1600-0889.2011.00556.x, 2011
  22. Groß, S., Esselborn, M., Weinzierl, B., Wirth, M., Fix, A., and Petzold, A., Aerosol classification by airborne high spectral resolution lidar observations, Atmos. Chem. Phys., 13, 2487–2505, 2013, doi:10.5194/acp-13-2487-2013, 2013.
  23. Gupta P. and Christopher S. A., Particulate matter air quality assessment using integrated surface, satellite, and meteorological products: 2. A neural network approach, J. Geophys. Res., 114, D20205, 2009
  24. Henriksen, T et al., Determination of soot refractive index as a function of height in an inverse diffusion flame, Proc. of 5th US Combustion Meeting, 1795-1803, 2007
  25. Hertz J. A. et al., Introduction to the Theory of Neural Computation, Addison-Wesley, Redwood City, Calif., 1991
  26. Hess, M., P. Koepke, and I. Schult, Optical properties of aerosols and clouds: The software package OPAC, Bull. Am. Meteorol. Soc., 79, 831 – 844, 1998.
  27. Hinton G. et al., A Fast Learning Algorithm for Deep Belief Nets, Neural Computation, 18(7), 1527 – 1554, 2006
  28. Jayne et al., Development of an Aerosol Mass Spectrometer for Size and Composition. Analysis of Submicron Particles, Aerosol Science and Technology 33:1-2, 49-70, 2000
  29. Koepke, P., M. Hess, I. Schult, and E. P. Shettle, Global aerosol data set, MPI Meteorologie Hamburg Report No. 243, 44 pp, 1997.
  30. Liu Z. et al., Extinction-to-backscatter ratio of Asian dust observed with high-spectral resolution lidar and Raman lidar, Appl. Opt., 41, 2760–2767, 2002
  31. Mishchenko, M. I., and L. D. Travis, Light Scattering by Nonspherical Particles, Academic Press, San-Diego, 690pp., 2000.
  32. Mona L. et al., Lidar Measurements for Desert Dust Characterization: An Overview, Advances in Meteorology, vol. 2012, Article ID 356265, 1-36, doi:10.1155/2012/356265, 2012
  33. Muller D. et al., Aerosol-type-dependent lidar ratios observed with Raman lidar, J. Geophys. Res., 112, D16202, doi:10.1029/2006JD008292, 2007
  34. Muller D. et al., Microphysical particle parameters from extinction and backscatter lidar data by inversion with regularization: simulation, Applied Optics Vol. 38, No. 12 20 April 1999(b)
  35. Muller D. et al., Raman lidar observations of aged Siberian and Canadian forest fire smoke in the free troposphere over Germany in 2003: Microphysical particle characterization, J. Geophys. Res., 110, D17201, doi:10.1029/2004JD005756, 2005
  36. Munoz, O., Volten, H., de Haan, J. F., Vassen, W., and Hovenier, J. W., Experimental determination of scattering matrices of randomly oriented fly ash and clay particles at 442 and 633 nm, J. Geophys. Res., 106, 22833–22844, 2001.
  37. Murayama T. et al., An intercomparison of lidarderived aerosol optical properties with airborne measurements near Tokyo during ACE-Asia, J. Geophys. Res., 108, 8651, doi:10.1029/2002JD003259, 2003
  38. NeuroSolutions software manuals
  39. Ng N.L. et al. An Aerosol Chemical Speciation Monitor (ACSM) for routine monitoring of the composition and mass concentrations of ambient aerosol, Aerosol Sci. Tech., vol.45, pp. 770–784, 2011a.
  40. Noh Y. M. et al., Seasonal characteristics of lidar ratios measured with a Raman lidar at Gwangju, Korea in spring and autumn, Atmos. Environ., 42, 2208–2224, doi:10.1016/j.atmosenv.2007.11.045, 2008
  41. Omar A.H. et al., The CALIPSO Automated Aerosol Classification and The calipso automated aerosol classification and lidar ratio selection algorithm, J. Atmos. Ocean. Tech., 26, 1994–2014, doi:10.1175/2009jtecha1231.1, 2009
  42. Papayannis, A., et al., Systematic lidar observations of Saharan dust over Europe in the frame of EARLINET (2000–2002), J. Geophys. Res., 113, D10204, doi:10.1029/2007JD009028, 2008
  43. Pappalardo G. et al., Four-dimensional distribution of the 2010 Eyjafjallajökull volcanic cloud over Europe observed by EARLINET, Atmos. Chem. Phys., 13, 4429-4450, doi:10.5194/acp-13-4429-2013, 2013
  44. Petit J.-E. et al. Two years of near real-time chemical composition of submicron aerosols in the region of Paris using an Aerosol Chemical Speciation Monitor (ACSM) and a multi-wavelength Aethalometer, Atmos. Chem. Phys., vol. 15, pp. 2985-3005, 2015. doi:10.5194/acp-15-2985-2015
  45. Priddy K. and Keller P., Artificial Neural Networks: An introduction, SPIE The International Society for Optical Engineering, Washington 98227-0010, USA, 2–5, 2005
  46. Sakai T. et al., Case study of Raman lidar measurements of Asian dust events in 2000 and 2001 at Nagoya and Tsukuba, japan, Atmos. Environ., 36, 5479–5489, doi:10.1016/S1352-2310(02)00664-7, 2002
  47. Schnaiter, M., H.Horvath, O. Möhler, K.-H. Naumann, H. Saathoff, O.W. Schöck, UV-VIS-NIR spectral optical properties of soot and soot-containing aerosols, J. Aerosol Science, 34, 1421-1444, 2003
  48. Sicard M. et al., Monitoring of the Eyjafjallajökull volcanic aerosol plume over the Iberian Peninsula by means of four EARLINET lidar stations, Atmos. Chem. Phys., 12, 3115-3130, doi:10.5194/acp-12-3115-2012, 2012
  49. Stohl, A., M. Hittenberger, and G. Wotawa, Validation of the Lagrangian particle dispersion model FLEXPART against large scale tracer experiment data, Atmos. Env., 32, 4245-4264, 1998.
  50. Taylor M. et al., Satellite retrieval of aerosol microphysical and optical parameters using neural networks: a new methodology applied to the Sahara desert dust peak, Atmos. Meas. Tech. Discuss., 6, 10955-11010, 2013
  51. Tesche, M., et al., Vertical profiling of Saharan dust with Raman lidars and airborne HSRL in southern Morocco during SAMUM, Tellus, Ser. B, 61, 144–164, 2009
  52. Tesche, M., et al., Profiling of Saharan dust and biomass-burning smoke with multiwavelength polarization Raman lidar at Cape Verde, Tellus (2011), 63B, 649–676, 2011
  53. Veselovskii I. et al., Application of randomly oriented spheroids for retrieval of dust particle parameters from multiwavelength lidar measurements, J. Geophys. Res., 115, D21203, doi:10.1029/2010JD014139, 2010
  54. Vladutescu D.V. et al, Urban aerosol backscatter properties based on Raman Lidar and parametric coefficients, Proceedings of the SPIE, Sandiego, CA, 2007, 66810E/1-10
  55. Wandinger U. et al., Optical and microphysical characterization of biomass-burning and industrial-pollution aerosols from multiwavelength lidar and aircraft measurements, J. Geophys. Res., 117, 8125, doi:10.1029/2000JD000202, 2002
  56. Wandinger U., A. Hiebsch, I. Mattis, G. Pappalardo, L. Mona, and F. Madonna, Aerosols and Clouds: Long-term Database from Spaceborne Lidar Measurements, Final report, ESTEC Contract 21487/08/NL/HE, 2011.
  57. Wandinger, U., and A. Ansmann (2002), Experimental determination of the lidar overlap profile with Raman lidar, Appl. Opt., 41, 511– 514, 2002
  58. Waterman, P.C., Matrix formulation of electromagnetic scattering. Proc IEEE 1965;53:805–12, 1965.