ATN DAC SYSTEM OVERVIEW:

The ATN DAC is an on line data distribution service for data retrieved from 
electronic tags attached to marine animals, as well as environmental data 
associated with those tag data. Data are supplied to the ATN DAC by animal 
tracking tag data researchers. The supplied data can be downloaded by anyone, 
within permission constraints set by the individual data providers. Some 
elements of data that are with withheld from direct download distribution may 
still be viewed graphically on line. Contact information for inquiries about 
withheld data are provided on the ATN DAC web interface.

...more to come (mergeing with ATNTechOverview.txt)


ATN DAC OPERATIONS:

Operations of the ATN DAC are divided into three broad categories: Web Handler,
Data Handler, and Security Handler groups. A handler is an operational element, 
(which may be hardware or software and may be automated or include human actions
in their operations), that performs a particular part of the sequence of events 
required to move information through their portion of the full ATN DAC system. 
Information may be in the form of automated or manual user inputs and responses, 
automated or manual data and metadata requests and responses, manual or 
automated data processing operations results, and manual or automated data 
ingestion operations results.

All handler operations are strongly interconnected. But, the three broad 
categories represent significantly different aspects of the overall data 
operations scheme, ranging from initial acquisition to final user download.

Handlers, with respect to operational codes, might be considered as analogous 
to metadata, with respect to data. They are processes that enable the overall
operations of the ATN DAC system, but, do not perform the majority of processing.

Web Handlers:

The web handler processing group elements are broadly defined as operational
components of the ATN DAC system that perform their tasks over the internet, 
though, individually, may or may not be directly associated with a web page. They
may be fully automated processes, (e.g., vendor API datastream operations), or 
involve manual inputs, (e.g., user login and file upload operations). 

The final web-based distribution of data is major purpose of the web handler
processing group, but web group operations are not limited to that. The ATN DAC 
has several web pages, including:

   main ATN DAC web page - http://oceanview.pfeg.noaa.gov/ATN/
   user registration page - https://mola.stanford.edu/ATN_Registration/
   user login and upload page - https://mola.stanford.edu/atnupload/ 
   metadata entry forms pages - https://mola.stanford.edu/metaforms
   private administration page - https://mola.stanford.edu/atnadmin

all of which have their own suite of web handlers, specific to page uses.

There are also multiple internet based operations that do not operate directly 
through a web page. For example, automated email notifications are sent to 
ATN DAC administrators (email: atnadmin@mola.stanford.edu) whenever users or
automated software perform a data handling or administrative operation that 
updates information for the ATN DAC system. The automated email system may be 
triggered by a web page entry, but, operates independently of the web pages so
is included separately as one of the web handler processing group elements.
All of the automated machine-to-machine data transfer operations are part of the
web handler processing group.


Data Handlers:

The data handler processing group far and away encompasses the largest 
operational segment of the ATN DAC system. Although some data handlers are
involved in user interactions with the ATN DAC system, the majority of data 
handler group operations are completely transparent to ATN DAC users. Data 
handler operations provide data to the ATN DAC for distribution, but, are not 
directly a part of the ultimate web-based data distribution stem. The main
data handler group operations are:

   data acquisition - any process which retrieves data from a user upload
                      repository or from a machine-to-machine transfer upload
                      repository. This is the acquisition step for the data 
                      processing system, not the (typically web-based) tag data 
                      retrieval operation.  

   format translation - any process which modifies the received format of data 
                        records to be compatible with the record format required
                        by a particular data processing step

   data processing - any process that takes properly formatted input data and 
                     performs operations on those data to analyze and modify
                     them for final distribution, e.g., quality control, 
                     statistical analysis, offset corrections, filtering, etc.
                      
   standards application - a subset of data processing ensuring correct units
                           are applied through out data process operations,
                           which includes date (timezone) operations and date 
                           format conversions when required, data value 
                           conversions to MKS when required, etc.

   data distribution - any process which moves processed data into the correct
                       repository to be picked up by the (typically web-based)
                       final data distribution system. These include ingestion 
                       of data records into the ATN DAC database, placement of 
                       data files in direct download directories, placement of 
                       (netCDF) data files in public and protected file-based
                       ERDDAP server repositories, etc. 


Security Handlers:

The security handler processing group includes all operations related to user
input authentication and data embargo protection. While the ATN DAC is primarily
an open access download system, it is necessary to keep track of individual user
data uploads for proper attributions, and, for some users, it is necessary to
prevent direct download of specified datasets through a data embargo system.

user registration and login security:

Users who wish to upload data using the file upload webpage, or submit metadata
through the metadata form pages must have an authenticated email address and
password. Users can have their email address authenticated by registering at the
ATN DAC registration web page. When a user registers at the ATN DAC registration
web page a unique ID code and a user specific upload directory are created for
them, and they recieve an automatically generated authentication code via the 
email address they provide while filling in the on-line registration form. After
they submit the authentication code on the registration web page, their 
registration is finalized and they can use the file upload web page and the
metadata form web pages. 

data upload security:

A user must log in before they can use the ATN DAC upload web page. When a user
provides files to the ATN DAC system through the upload web page, the uploaded 
files are placed within a data directory set up for their use. The files are 
secured behind multiple firewalls within a filesystem that even the user who
uploaded the data does not have access to. Automated data handler group 
processes scan the upload directories for new information and direct the
incoming information to the proper data processing systems.

A user must provide an authorized email address to submit information through 
the ATN DAC metadata form web pages. When a user submits metadata via the 
metadata form web pages, that information is passed to the ATN DAC system via 
automated email processes. Users do not have direct access to the automated
email system.

data distribution security:

The primary ATN DAC data distribution systems include protected and unprotected
ERDDAP servers, direct file download links, and database accesses. While 
unembargoed data are not protected from web-based distribution, all underlying
data distribution systems, whether for public access data or embargoed data, are
protected from direct user access. When a user requests data that request is
processed through administrator only access operations that accept user 
requests, perform the necessary data system accesses, then return the requested
data or data access links to the user, acting as a go between from the user to 
the data processing and storage systems.

Embargoed data have a further level of protection, where no data are directly
plotted on the main ATN DAC web portal, rather, only images of the current data
requested are provided, with no position information, and no data download 
links provided. Access to embargoed data can be had only by request to the data
provider through contact information provided on the ATN DAC web site.