OiDB User Manual

Revision : 0.5

Date : 15 June 2015

Authors : JMMC oidb Working Group

Table of Contents

1. Introduction

2. DataPIs

3. Terms of use: USER MUST READ

All parties must agree and comply with the following terms of use before using the database. The cookie at the very bottom of the page is meant to officially accept these terms and also to provide the OiDB team with statistics to improve the quality of the service.

"This research has made use of the Jean-Marie Mariotti Center \texttt{OiDB} service \footnote{Available at http://oidb.jmmc.fr }."

4. Content of the database

Data reduction level Content What is stored in OiDB Access Policy
Level 0 (L0) Raw Metadata only (meaning Observation logs) Public
Level 1 (L1) Reduced uncalibrated None (not offered in OiDB 1.0) -
Level 2 (L2) Reduced calibrated Data and metadata Public, Embargo period possible for PIONIER data*
Level 3 (L3) Published Data and metadata Public

*PIONIER L2 data are regularly uploaded to OiDB in an automatic way. Following the conditions set by the PIONIER team, they become publicly available for download one year after the observations took place. For any enquiry, please contact Dr. Jean-Baptiste LeBouquin ( Jean-Baptiste.Lebouquin AT obs.ujf-grenoble.fr).

5. Granularity and Data Model

1 granule = 1 target / 1 night / 1 instrument mode / 1 OIFITS

The metadata of a granule will often represent a subset of an OIFITS metadata. This implies that several granules might correspond to the same OIFITS file.

6. Diffusing data with OiDB ("Submit new data" menu)

New dataPIs have to click on "Register" to provide JMMC with a name and an email address. WARNING: that email address will be given to any user interested in the dataPI's data. It is therefore important to choose this address carefully. Some people/groups in charge of an instrument might want to use a generic address here (e.g. instrument@facility.country) rather than their own personal email address. On the search result panel, the dataPI's email address is hidden behind a "Mail-to" icon in order to avoid spamming.

Once the dataPI is logged in four submission options are available:

1) From OIFITS files: Calibrated (L2) and Published (L3) OIFITS

A fast 3-step process allows dataPIs to add OIFITS to the database.

First, a syntactic test is performed on the files to check whether they are valid OIFITS files or not. If the test is positive, the message "Successfully uploaded" appears in green after clicking the "Add" button. Otherwise, a useful error message is printed. Zipped files with extension ".zip" are supported (all files must be at the root level). Regarding the name of the file, note that that the following ASCII signs ' #%:?[]` will be transformed into "%XX" (e.g "Test:MIRC.oifits" will be recorded as "Test%3FMIRC.oifits" into the database). Non-ASCII will be also replaced by "%XX%XX". So they better be avoided. The sign "/" is forbidden.

Secondly, next to the name of the file, a blue "information" button lists the syntaxic information, warnings and errors that occurred during the upload. The gear icon allows dataPIs to examine the OIFITS with an application such as OIFITSExplorer via the SAMP protocol (see section 8 of this page). If the previous steps were successful, metadata are extracted and a set of granules (one per line) is created. The dataPI then checks and completes a pre-filled short form to validate and add the metadata that are not included in the OIFITS file but which are fundamental to describe the data (namely the instrument mode, the program ID of the observations and a quality flag).

Upload information:
Information_button.png

* The "Program ID" field corresponds to any identifying number of the observing program that led to the data the dataPI submits. It is given by the observatory where the observations were executed (e.g. "94.D-0074" or "94.D-0074(A)" for ESO and "CLASSIC 10" for CHARA). This information is worth-indicating to describe the data (data curation) and eventually provide some external links (e.g. ESO Observation Schedule page for a given run).

* The "Target" column displays the Target name and coordinates taken from OI_Target. Based on the coordinates, close objects found with SIMBAD are suggested in case the Target name and coordinates were approximated.

* "Coordinates": close objects found by SIMBAD are suggested in the scrolling menu in case the coordinates were approximated. In future versions, for the L2 level, coordinates will have to be valid. If coordinates are null, the OIFITS submission will be rejected (SEVERE error). The name of the object is informative (in the sense that it is not checked).

* For the "Instrument" and "Instrument mode" fields, suggestions are presented in scrolling menus. These were made to have a coherent sorting of data in the whole database. Please use preferably the suggestions listed under "Facilities and instruments" for the "Instrument" field. If you do so, the field box will turn green and a list of registered instrument modes will be suggested in the "Instrument mode" field next to it. Select the one that corresponds to the given granule.

* Quality flag: since there is no magical formula to estimate the quality and scientific usefulness of a granule, a quality flag allows the dataPI to mark the overall quality of a granule using the following grades:

Unknown (default)

Trash

To be reduced again

Quick look (risky to publish)

Science ready

Outstanding quality

2) From a collection of OIFITS files on VizieR: interferometric data reported in a publication may be submitted as a VizieR catalog. Once it is done, the data can be automatically imported to OiDB by giving the name of the catalog. Similarly to the previous submission mode, a form that summarizes the main metadata allows the user to modify Target, Instrument and Instrument mode before validation.

3) From a metadata file: OIFITS files may be alternatively locally processed for metadata extraction and converted to an XML document. This document can be refined with data missing from the source files prior to being uploaded to a special endpoint of the service. Currently this technique is employed for batch processing and uploading the PIONIER L2 dataset to the database. This way of submitting metadata is suitable for large collection of OIFITS files, restricted access files and also non-OIFITS files (provided a suitable parser/extraction tool is written by the dataPI).

4) From an external database/service (for administrators): this submission mode is used by administrators to upload L0 data.

Once a set of OIFITS files has been successfully submitted, metadata are stored in tables that follow the ObsCore Data Model. OIFITS are only read and are not modified by our service so that they remain exactly as they have been submitted.

7. Querying data (Search)

8. Operating data

* Results panels 1:

* Results panels 2:

* Results panel 3:
res_panel_3.png

9. License

The portal and all its contents including all sorts of data are protected under the Creative Commons License "Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)". Information can be found here.

10. References

[1] Haubois, X., Bernaud, P., Mella, G. et al., 2014, Proc. SPIE 9146, Optical and Infrared Interferometry IV, 91460O (24 July 2014); doi: 10.1117/12.2056977

[2] Pauls, T.A., Young, J.S., Cotton, W.D., & Monnier, J.D., “A Data Exchange Standard for Optical (Visible/IR) Interferometry”, PASP, 117, 1255, (2005)

[3] Louys, M., Bonnarel, F., Schade, D., et al., “IVOA Recommendation: Observation Data Model Core Components and its Implementation in the Table Access Protocol Version 1.0”, arXiv:1111.1758, (2011)

[4] Williams, R., Hanisch, R., Szalay, A., & Plante, R., “IVOA Recommendation: Simple Cone Search Version 1.03”, arXiv:1110.0498, (2011)

[5] Ortiz, I., Lusted, J., Dowler, P., et al., “IVOA Recommendation: IVOA Astronomical Data Query Language Version 2.00”, arXiv:1110.0503, (2011)

[6] Dowler, P., Rixon, G., & Tody, D., “IVOA Recommendation: Table Access Protocol Version 1.0”, arXiv:1110.0497, (2011)

[7] Taylor, M.B., Boch, T., Fay, J., Fitzpatrick, M., & Paioro, L.,“SAMP: Application Messaging for Desktop and Web Applications”, Astronomical Data Analysis Software and Systems XXI, 461, 279, (2012)

[8] Lafrasse, S., Bourges, L., & Mella, G.,”SAMP App Launcher - An on-demand VO application starter by JMMC”, Astronomical Data Analysis Software and Systems XXI, 461, 379, (2012)

[9] Tallon-Bosc, I., Tallon, M., Thiébaut, E., et al,”LITpro: a model fitting software for optical interferometry”, Proceedings of the SPIE, 7013, (2008)