DAP4Cor: An OPeNDAP server for CORIOLIS data centre

This slide deck was presented as part of the OPeNDAP Developer's Workshop, Winter 2007 that was held from February 21-23, 2007 at the NCAR Foothills Laboratory in Boulder, CO. 

Abstract: 

CORIOLIS is an operational global ocean in-situ data centre. As such, CORIOLIS collects, does quality assessment and disseminates datasets. The vertical profiles datasets are fully managed in an Oracle database. In order to answer operational data centres (mainly ocean forecasting centres) needs, CORIOLIS extracts on a regular basis many netCDF files from the database. The datasets are usually in a standard format (ARGO) but, depending on the users needs, the contents and update frequencies differ. The netCDF files are made available through an ftp server.

In order to avoid maintaining so many extraction scripts and replications of datasets in netCDF files and to improve the data center interoperability, we have decided to set up an OPeNDAP interface on the CORIOLIS database. For internal usage it centralizes the extraction facility, for external users it gives a standard interface for interactive or batch extraction.

After studying the DAPPER solution from PMEL, it has been decided to keep the DAP profile data model but to develop a specific implementation based on our Oracle database.

The server called DAP4COR is a web application running under tomcat. It is based on the java DAP 1.1.7 API. The most of work has been spent by, on one hand translating OPeNDAP requests criteria into SQL requests, on the other hand setting up a profile interface which reads the database tables and serializes into a DAP response.

For accessing the server, a pool of MATLAB scripts and a Dapper to netCDF conversion tool have been developed. The further developments will aim at improving the server delays of response and integrating, in addition to vertical profiles, moorings and trajectories data type (following the dapper observation data convention).

Server : http://www.ifremer.fr/dap4cor Documentation and tools : ftp://ftp.ifremer.fr/ifremer/coriolis/tools/dap4cor

AGENDA OF THE WORKSHOP 

[Almost all of the presentation listed here are available in the OPeNDAP Community Zenodo library.] 

Wednesday February 21, 2007

Session 1

0900-1000 OPeNDAP Current Status

• Introduction by Peter Fox, President of OPeNDAP

1000-1015 Break

Session 2 (moderator: P. Fox)

1015-1200 The OPeNDAP 4 Data Server (aka Hyrax)

• James Gallagher and Nathan Potter - Customizing and Extending Hyrax
• Patrick West - Harnessing the power of the Server 4 Back-End Server modular framework
• Jose H. Garcia - Numerical Grid Computations with the OPeNDAP Back End Server (BES)
• Patrick West - OPeNDAP Server 4 Back-End Server Authentication and Authorization

Lunch 1215-1330

Session 3 (moderator: P. Cornillon)

1330-1530 Security

• John Relph and Kenneth S. Casey - Software Development and Security at NOAA
• John Caron - The THREDDS Data Server and OPeNDAP security
• Tim Pugh - BMRCs OPeNDAP data service, how far can it reach?
• Discussion

1530-1545 Break

Session 4 (moderator: J. Gallagher)

1545-1730 Semantics

• Benno Blumenthal- Using an RDF framework to carry metadata for datasets
• Rob Raskin - SWEET - an upper-level ontology for Earth and Space Sciences
• (withdrawn) Luis Bermudez - MMI and CF
• Peter Fox - OPeNDAP and semantics
• Discussion

Dinner

No-host dinner, Cafe Gondolier

Thursday February 22, 2007

Session 5 (moderator: D. Holloway)

0900-1030 APIs and Clients

• Roberto De Almeida - Data Access Protocol meets Python
• Peter Cornillon - The Matlab OPeNDAP GUI Suite

1030-1100 Break

• Poster: Denis Nadeau - Enhancing Access to NASA Satellite Data OGC Web Services using OPeNDAP.

Session 5 continued

1100-1215 APIs and Clients (continued)

• John Chamberlain - OPeNDAP: User Versus Programmatic Access
• Greg Janee - Data Discovery in a Distributed Environment and Darren Hardy's AGU presentation.

Lunch 1215-1315

Session 6 (moderator: P. Cornillon)

1315-1445 Data Providers

• Roy Mendelssohn - OPeNDAP at ERD with suggestions for future development
• Jim Potemra - IPRC data transport, LAS and EPIC server
• Christopher Lynnes - OPeNDAP Developments at the Goddard Earth Sciences DISC
• Thomas LOUBRIEU - OPeNDAP in European oceanography data management

Session 7 (moderator: N. Potter)

1445-1715 Server-Side Operations with 1530-1600 Break

• Wenli Yang - The ROSES ACCESS OPeNDAP/OGC Gateway Project
• Roland Schweitzer - Server-side OPeNDAP Analysis - A General Approach Utilizing Legacy Applications through TDS
• Daniel L. Wang - Server-side Data Reduction and Analysis with Script Workflow Analysis for MultiProcessing (SWAMP)
• Thomas LOUBRIEU - Dap4cor: A Dapper-like server for CORIOLIS in-situ data centre

Dinner

Dinner on your own

Friday February 23, 2007

Session 9 (moderator: P. Fox)

0900-0930 Server-side functions (continued)

• James Gallagher - Server-side Functions for Geo-spatial Selection

0930-1200 Strategies and Directions

• Discussion (with nominal time allocations) focusing on...
• community forums for deciding on standards processes within (and beyond) the OPeNDAP community, who is involved, timeframe, prototypes, etc. (1hr)
• security (20min)
• semantics (profiles, vocabularies) (20min)
• server-side (aggregation) (20min)
• response types (e.g. get_capabilities) (20min)
• metrics (10min)
• others: gateways, Server4, TDS and other general direction issues

Lunch

Hyrax Tutorial (FL2/1001): 1300-1630

Dinner

No-host dinner, location TBA