GUK

DCS Aspect Normal

Krik krik krik

 GUK
The South Pole CARA Project:
A DCS demonstration
• A data cycle system was produced by the RIT consortium for the
CARA project (Harper, PI) at the South Pole.
• The South Pole DCS has the following history:
– The South Pole IR telescope operated for 5 years as a PI project.
– NSF indicated that the project needed to make telescope time with
their camera publicly available.
– NOAO agreed to run the proposal selection.
– NOAO selected 21 proposals, 13 of which were actually
implemented.
– Gatley, a member of the CARA consortium, and his colleagues at
RIT developed and implemented the data cycle pipeline for this
experiment.

March 7-8 2000 DCS Preliminary Design Review 2

 GUK
South Pole DCS:
Implementation
• Both raw and calibrated data were provided to the PIs.
• Data were archived so they could be used for further global
processing
– For example, Kastner kept track of L-band photometry in order to
assess the site's performance at this band.
• Some 6 - 8 months of observations (IR imaging) took 1 month of
processing at RIT.
• The process was not optimized, and it was written only for the CARA
mode of observing.
• Rhody and Gatley spearheaded the project.
• Kastner (who planned to use the data, and who came to RIT as a
faculty member in July 1999) improved and refined the process by
interacting with Rhody, and by actually using the pipeline.
March 7-8 2000 DCS Preliminary Design Review 3
 GUK
South Pole DCS: Philosophy
• Commercial software tools were used whenever possible.
• Software written by the consortium defines the interaction
between tools and/or packages and the data.
• In addition, standard packages employed in other astronomy
pipelines were used.
• Examples of commercial software include:
– XML (extensible markup language);
– CORBA (Common Object Resource Broker Architecture);
– IDL Lib_Astro;
– JAVA;
– SQL (a database query language)

March 7-8 2000 DCS Preliminary Design Review 4

 GUK
South Pole DCS:
Portability and Scalability
• Because this was a straightforward application, Harvey Rhody
had all of the DCS for CARA on his computer at RIT.

• The system is portable, and could be configured to run on

several resident computers with balanced loads or on separated
computers over the internet.

• This experiment represents a proof of concept:

– the RIT consortium has designed and executed a DCS.

March 7-8 2000 DCS Preliminary Design Review 5

 GUK
SOFIA DCS History
• HAWC presented their pipeline designs to SOFIA
because:
– Gatley and Harper (the South Pole DCS principals) were also
principals in the HAWC experiment,
– Harper (PI HAWC) proposed an end-to-end design,
– Gatley and Harper had already teamed to successfully implement the
data pipeline for the CARA project.
• Becklin was interested in a common pipeline approach for all the
SOFIA instruments
• Since RIT already had demonstrated competence, they were
asked to describe plans for a DCS to SOFIA

March 7-8 2000 DCS Preliminary Design Review 6

 GUK

Conceptual Design Review

(June 1999)

March 7-8 2000 DCS Preliminary Design Review 7

 GUK
The CoDR material is on the
web
• http://sofia-usra.arc.nasa.gov/internal/science/DCS/dcs_cdr.html

• Username = science

• Password = sofia

March 7-8 2000 DCS Preliminary Design Review 8

 GUK

The DCS Mission

The DCS shall provide the infrastructure to support the
operation of facility instruments on SOFIA using best
current knowledge and tools, and supporting
continuous improvement in an efficient, extensible,
and modular architecture.

http://sofia-usra.arc.nasa.gov/internal/science/DCS/dcs_cdr.html

March 7-8 2000 DCS Preliminary Design Review 9

 GUK

Results of Conceptual Design

Review

March 7-8 2000 DCS Preliminary Design Review 10

 GUK

Need for DCS

The CoDR committee finds that a DCS is necessary for making the facility
instruments on SOFIA accessible to the largest possible astronomy
community.

Therefore, the Director should consider the development of the DCS of

equivalent importance to the development of the FSIs.

March 7-8 2000 DCS Preliminary Design Review 11

 GUK
Recommendations of CoDR
Committee
1. The USRA should commission the development of a DCS,
and the first priority for the DCS is the recording and
archiving of all data necessary for complete analysis,
calibration and future access to observational data obtained
with the FSIs.

2. In order for the FSIs to be readily accessible to the

community of US astronomers, the second priority for the
DCS is to establish a select number of observing modes
supported by pipeline reduction, including quicklook analysis,
for each of the FSIs.
March 7-8 2000 DCS Preliminary Design Review 12
 GUK
Recommendations of CoDR
Committee
3. The Director should enable a close working relationship
between the DCS team(s) and the FSI teams.

4. The DCS should become the vehicle for providing on-line

documentation and information necessary for the GI to
propose and carry out observations with the FSIs.

5. The DCS should be implemented as a system of distributed

modules, integrated by an extensible hardware and software
infrastructure, as presented at the DCS Conceptual Review.
March 7-8 2000 DCS Preliminary Design Review 13
 GUK
Top Level Requirements
• The recommendations of the CoDR
committee ARE the top level
requirements for the DCS

March 7-8 2000 DCS Preliminary Design Review 14

 GUK
Top level requirements (1)
• The DCS shall record and archive all
data necessary for complete
analysis, calibration and future
access to observational data
obtained with the FSIs.
• The DCS shall support a select
number of observing modes with
pipeline reduction, including
quicklook analysis, for each of the
FSIs.
March 7-8 2000 DCS Preliminary Design Review 15
 GUK
Top level requirements (2)
• The DCS shall provide on-line
documentation necessary for the GI
to propose and carry out
observations with the FSIs.
• The DCS shall be implemented as a
system of distributed modules,
integrated by an extensible hardware
and software infrastructure.
March 7-8 2000 DCS Preliminary Design Review 16
 GUK
The DCS in plain English
• The program architecture planned consists of "dumb" modules
(e.g. the parser) that need descriptions of what to do, and where
and how to find the appropriate tools.
• Scripts are written which are particular to the data-taking mode
for the instrument, and which utilize these tools on the data.
• All of these components are also modular.
• The scripts can be written by RIT (or the PI).

March 7-8 2000 DCS Preliminary Design Review 17

 GUK
The CARA example
• Experiments “scripted” for execution
• Observations made by the operator (Charlie Kaminsky).
– Standard modes of operation, common standards
• Data sent by satellite to White Sands and then by Internet.
• Dark-frame subtraction, linearization, flat-fielding, co-addition,
background-subtraction etc., performed as required, using IDL.
– Saved the method used, the raw data, and the final product
• intermediate products not generally retained;
• intermediate products from some point in the path can be retained if
desirable
• Images calibrated.
• Data written in the desired format*.
* There would be a stage to put the product into IPAC archival form for SOFIA
• Data archived.
• Data products disseminated.
March 7-8 2000 DCS Preliminary Design Review 18
 Overview of the PDR
• The "Core Development" DCS has been provided priorities by the June
'99 Conceptual Design Review.
• This core development must have the capability to capture the data,
reduce it, and provide documentation, all within a modular infrastructure
utilizing extensible software/hardware tools.
• The DCS consortium will present their view of the required large
components, and the software tools and programs needed to get the
components to work together efficiently in the pipeline.
• The DCS consortium will also present the collected requirements for
support of extensibility, including (presently unfunded, but envisaged):
• expanded scope for facility instrument support;
• additional participation (universities, ARC, GSFC, and IPAC);
• tools for USRA FSI flight planning;
• tools for USRA to manage the request for general investigator telescope time;
• tools for visualizing the use of SOFIA facility instrumentation;
• unified approach to USRA control of the facility instruments;
• Support of PI instruments.
 GUK
Interface Management and
Continuous Improvement
• With Facility Science Instruments
• With MCCS
• With Archives (IPAC, etc)
• With General Investigators

• Supporting change and continuous improvement

– The strength of SOFIA and the fundamental
difference between this and the other Great
Observatories.

March 7-8 2000 DCS Preliminary Design Review 20