3. ISTP Global Attributes¶
Global attributes are used to provide information about the dataset as an entity and about the individual files composing the datasets. Together with the variables and variable attributes, the global attributes make the data correctly and independently usable by someone not connected with the instrument team, and hence a good archive product. The global attributes are also used by the Coordinated Data Analysis Web (CDAWeb) data browsing and retrieval system hosted by SPDF, and by other data display and analysis software such as AutoPlot and SPEDAS.
The required, recommended, and optional global attributes are listed in the table below. See Global Attribute Definitions for the full set of defined global attributes in alphabetical order; however, note that global attributes in a file can be listed in any order. Also note that the attribute names are case sensitive, and the names of the ISTP global attributes must match the case exactly as shown. Additional global attributes may be defined, but their names must start with a letter and contain letters, numbers, and underscore character but no other special characters. Though attribute names are case-sensitive, the names must not be distinguished by case only.
All ISTP global attributes are of character data type (CDF_CHAR) in the CDF file format. Since the CDF format allows multiple entries for global attributes, with each ISTP global attribute entry being a 1-D array of CDF_CHAR (a string), this allows storing a 1-D array of strings in a global attribute. Note that these multiple entries for an ISTP global attribute in the CDF format correspond to a 1-D array of string data type in the netCDF-4 format.
Attribute Name |
Requirement |
Example Value |
Notes |
|---|---|---|---|
Required |
|
Identifies the data type of the dataset. Both the short and long names, separated by |
|
Required |
|
Identifies the version of a particular data file. |
|
Required |
|
Identifies both the short and long names, separated by |
|
Recommended, required by CDAWeb (allows multiple entries; five entries allowed by CDAWeb) |
|
Combines types of instruments and regions as a simple way to quickly separate types of datasets for search through CDAWeb. CDAWeb allows up to five entries from |
|
Required |
|
Stores the name of the file using the ISTP naming convention |
|
Required |
|
Carries |
|
Required |
|
Writes out the full names associated with the encrypted |
|
Recommended, required by CDAWeb |
|
Has a single value and is used to facilitate making choices of source through CDAWeb. |
|
Required |
|
Lead organization, usually that of the Principal Investigator (PI); should include a recognizable abbreviation. |
|
Required |
|
Lead person, usually Principal Investigator (PI); should at least include first initial and last name. |
|
Required |
|
Identifies the mission or investigation that contains the sensors. Both the short and long names, separated by |
|
Required (allows multiple entries) |
|
Contains description of the key measurements provided by the dataset. A short description of the experiment, with reference to a journal paper and/or web page describing the experiment in detail, is also essential. This attribute can have multiple entries (two entries are shown in the example). Used by CDAWeb as the dataset description. |
|
|
Required |
|
Start UTC date/time (in yyyy-mm-ddThh:mm:ss format) of the data in the file. |
|
Required |
|
End UTC date/time (in yyyy-mm-ddThh:mm:ss format) of the data in the file. |
Recommended |
|
Specifies the expected acknowledgement text if the dataset is cited. |
|
Recommended (allows multiple entries) |
|
Describes both the science discipline and subdiscipline. This attribute allow multiple entries. |
|
|
Recommended |
|
Describes the dataset processing level. Both the short and long names, separated by |
Recommended |
|
Digital Object Identifier (DOI) as a persistent identifier for the dataset. |
|
Recommended |
|
Describes the dataset file naming convention based on combination of |
|
Recommended |
|
Allows for the generating data center/group to be identified. |
|
Recommended |
|
Date of the file creation using yyyymmdd format. |
|
Recommended (allows multiple entries; five entries allowed by CDAWeb) |
|
Stores the URL for the link. Used with |
|
Recommended (allows multiple entries; five entries allowed by CDAWeb) |
|
Stores the text describing online data or documents. Used with |
|
Recommended (allows multiple entries; five entries allowed by CDAWeb) |
|
Stores the title of the web page holding online data or documents. Used with |
|
Recommended (allows multiple entries) |
|
Stores the history of modifications made to the dataset. This attribute can have multiple entries. |
|
Recommended (allows multiple entries) |
|
Identifies the name of the project, and indicates ownership. Both the short and long names, separated by |
|
Recommended |
|
Text containing information on citability and PI access restrictions. This may point to a web page specifying the rules of use. |
|
Recommended |
|
Unique dataset identifier assigned by a SPASE naming authority. |
|
Recommended |
|
Specifies time resolution of the data. |
|
Optional (allows multiple entries) |
|
Lists the parent files of derived and merged datasets. This attribute allows multiple entries. |
|
Optional, required by Cluster |
|
Version of the skeleton file used to create the data file. |
|
Optional, required by Cluster |
|
Version of the software that generated the data file. |
|
Optional |
|
Dataset title. |
|
Optional |
|
Written by software for automatic validation of features such as the structure of the data file on a simple pass/fail criterion. |
3.1. Global Attribute Definitions¶
3.1.1. Acknowledgement¶
(Recommended.) Specifies the expected acknowledgement text if the dataset is cited.
3.1.2. ADID_ref¶
(No longer relevant.) This attribute stores the control authority identifier associated with the detached SFDU label. If no control authority identifier has been assigned, then the identifier associated with the ISTP/IACG Guidelines (NSSD0241) or with CDF (NSSD0110) can be used.
3.1.3. Data_processing_level¶
(Recommended. PROPOSAL ONLY.) Processing level of the dataset. Both a short name and a long name, separated by >, are required. For example, Data_processing_level = "L2>Level 2"
3.1.4. Data_type¶
(Required.) This attribute identifies the data type of the dataset, generally the characteristics of the dataset that distinguish it from other datasets from the same instrument (including plausible ones created later) using some meaningful combination of parameters, resolution, format, compression. Both a short name (used in Logical_source and in the filename) and a long name (used in Logical_source_description), separated by >, are required.
If desired, the level (L0, L1, L2, key parameter kp, summary parameter sp, etc.) may be prepended to the data collection name to sort by these levels (such as l2-gms-62ms). Temporal resolution uses time codes such as: ms, min, s, hr, day, week, month; for example, 500ms for 0.5sec resolution, 6s for 6sec, 5min for 5 minutes, 2hr for 2 hour, 1day for daily.
Originally, for ISTP exchangeable data products, the values of Data_type were "Kn>Key Parameter" for approximately minute averaged survey data, "Hn>High Resolution data" for certified data of higher resolution than Key Parameters, and "Mn>Modified Data" for modified or derived datasets, wheren is from 0 to 9. For Cluster/Cluster Science Data System (CSDS), the allowed values were either "SP>Summary Parameter" or "PP>Prime Parameter". For new datasets, data providers are not restricted to these original definitions, and can define data types individually for each mission or instrument to capture relevant information. For example, Data_type = "L2-Summary>level 2 summary" indicates both the processing level (L2) and that the dataset contains summary data, as two subfields separated by hyphen.
3.1.5. Data_version¶
(Required.) This attribute identifies the version of a particular data file for a given date, e.g., file ge_k0_mgf_19920923_v01.cdf is the first version of data for 1992 September 23. Data_version generally starts at "1". Each time the data file is reproduced - for recalibration or other reasons - the Data_version is incremented by unity. Some projects use a more complex versioning scheme, such as Release, Major, Minor versions separated by period (e.g., Data_version = "2.13.004").
Note that the version, in case of a simple versioning, or the major version, in case of a more complex scheme, should be consistent across all files in the dataset (same version number have the same meaning for all files). This enables easy and unambiguous identification of major processing version for each data file. Description of all significant changes corresponding to each major processing version (up to, and including, the file current version) should also be included as a MODS global attribute entry (one entry per major version).
3.1.6. Date_Start, Date_End¶
(Required for netCDF only.) The two attributes, always used together, describe the start and end UTC date/time (in yyyy-mm-ddThh:mm:ss format) of the data in the file. Required for netCDF since netCDF does not have a standard or defined variable for epoch/time values. Used by CDAWeb.
3.1.7. Descriptor¶
(Required.) This attribute identifies the name of the instrument or sensor that collected the data. Both a short name (used in Logical_source and in the filename) and a long name (used in Logical_source_description), separated by >, are required. For example, Descriptor = "ISOIS>Integrated Science Investigation of the Sun". This attribute should have single entry.
3.1.8. Discipline¶
(Recommended.) This attribute describes both the science discipline and subdiscipline. Multiple entries are allowed. The list for space physics is:
"Solar Physics>Heliospheric Physics""Space Physics>Interplanetary Studies""Space Physics>Magnetospheric Science""Space Physics>Ionospheric Science""Space Physics>Astrophysics Science"
3.1.9. DOI¶
(Recommended.) Digital Object Identifier (DOI) as a persistent identifier for the dataset, of the form
https://doi.org/PREFIX/SUFFIX with the PREFIX identifying the DOI registration authority and the SUFFIX identifying the dataset. The DOI should point to a landing page for additional information about the dataset. DOIs are typically created by the SPASE naming authority or archive.
3.1.10. File_naming_convention¶
(Recommended.) Describes the dataset file naming convention using combination of "source", "descriptor", and "datatype" (correspond to short names in Source_name, Descriptor, and Data_type attributes) and a date/time format, all separated by "_", e.g., File_naming_convention = "source_descriptor_datatype_yyyyMMdd". Note that "source_descriptor_datatype" is the recommended order for new datasets, while "source_datatype_descriptor" may be common for older datasets. Date/time format includes a date format, either "yyyyMMdd" (recommended) or "yyyyDDD", possibly followed by a time format ("HHmmss", "HHmm", or "HH") with optional "t" separating them . E.g., File_naming_convention = "source_descriptor_datatype_yyyyMMddtHHmmss".
3.1.11. Generated_by¶
(Recommended.) This attribute allows for the generating data center/group to be identified.
3.1.12. Generation_date¶
(Recommended.) File creation date using yyyymmdd syntax, e.g., Generation_date = "19920923". This is distinct from the date in Validate below which records the times of later validation processes.
3.1.13. Instrument_type¶
(Recommended, required by CDAWeb.) Combines types of instruments and regions as a simple way to quickly separate types of datasets for search through CDAWeb. Up to five entries are allowed. The following list contains allowed values:
"Activity Indices""Dust and Debris""Electric Fields (space)""Engineering""Ephemeris/Attitude/Ancillary""Ground-Based HF-Radars""Ground-Based Imagers""Ground-Based Magnetometers, Riometers, Sounders""Ground-Based VLF/ELF/ULF, Photometers""Housekeeping""Imaging and Remote Sensing (ITM/Earth)""Imaging and Remote Sensing (Magnetosphere/Earth)""Imaging and Remote Sensing (Sun)""Magnetic Fields (Balloon)""Magnetic Fields (space)""Particles (space)""Plasma and Solar Wind""Radio and Plasma Waves (space)"
3.1.14. LINK_TEXT, LINK_TITLE, HTTP_LINK¶
(Recommended.) These attributes are used together to store the text (in LINK_TEXT, optional, even if the other two attributes are present), the title (in LINK_TITLE), and the URL (in HTTP_LINK) of the link to the related and useful resources and documents, including links to the online data available at the PI or Co-I web site. The links will be displayed on the CDAWeb Data Explorer page for the dataset. For example, if all three attributes are used (and each attribute includes one entry):
LINK_TEXT = "3-sec MGF magnetic field 1 Sep 1993 through 30 Sep 2015 available at "
LINK_TITLE = "ISAS DARTS"
HTTP_LINK = "https://www.darts.isas.jaxa.jp/stp/geotail/"
CDAWeb Data Explorer page for the dataset will display:
3-sec MGF magnetic field 1 Sep 1993 through 30 Sep 2015 available at ISAS DARTS
If only LINK_TITLE and HTTP_LINK are used:
LINK_TITLE = "ISAS DARTS"
HTTP_LINK = "https://www.darts.isas.jaxa.jp/stp/geotail/"
CDAWeb Data Explorer page for the dataset will display:
These attributes can have multiple entries (up to five entries allowed by CDAWeb), but exactly the same number of entries is required for each of the included attributes, with empty strings allowed. This will result in multiple lines of text displayed on the CDAWeb Data Explorer page for the dataset.
3.1.15. Logical_file_id¶
(Required.) This attribute stores the name of the data file, without extension, using the ISTP naming convention (Source_name_Descriptor_Data_type_Date_vData_version), with short names from Source_name, Descriptor, and Data_type attributes, e.g., Logical_file_id = "psp_isois_l2-summary_20180928_v07". This attribute stores the filename in case of an inadvertent filename change.
3.1.16. Logical_source¶
(Required.) This attribute carries dataset ID (used by CDAWeb) in the form Source_name_ Data_type_Descriptor, e.g., Logical_source = "psp_isois_l2-summary".
3.1.17. Logical_source_description¶
(Required.) This attribute writes out the full names associated with the encrypted Logical_source, e.g., Logical_source_description = "Parker Solar Probe ISOIS level 2 summary". Used by CDAWeb as the dataset full name.
3.1.18. Mission_group¶
(Recommended, required by CDAWeb.) This attribute has a single value and is used to facilitate making choices of source through CDAWeb. Examples:
"Geotail""GOLD""ICON""IMP8""Parker Solar Probe""Solar Orbiter""THEMIS""Wind""Van Allen Probes (RBSP)""Geosynchronous Investigations""Ground-Based Investigations""Helio ephemeris""OMNI (Combined 1AU IP Data; Magnetic and Solar Indices)""Smallsats/Cubesats""Sounding Rockets"
3.1.19. MODS¶
(Recommended.) This attribute is used to store the description of all significant changes to the dataset. The attribute is not directly tied to Data_version, but each version produced will contain relevant modifications, and description of each major version (including version change date and all significant changes) should be included as a MODS attribute entry (one entry per major version). This attribute can have unlimited number of entries to store the complete history of modifications made to the dataset.
3.1.20. Parents¶
(Optional.) This attribute lists the parent files (in CDF or other format) for derived or merged datasets. Multiple entries are used to include multiple parents. The Parents entry syntax, e.g., for a CDF parent file, is of the form"CDF>logical_file_id". Alternatively, the whole file name, including extension, can be included as a Parents attribute entry.
3.1.21. PI_affiliation¶
(Required.) Affiliation of the lead person, usually Principal Investigator (PI). This attribute value should include a recognizable abbreviation.
3.1.22. PI_name¶
(Required.) Lead person, usually Principal Investigator (PI). This attribute value should at least include first initial and last name.
3.1.23. Project¶
(Recommended.) This attribute identifies the name of the project and indicates ownership and funding project. For the original ISTP missions and investigations, the value used is "ISTP>International Solar-Terrestrial Physics". For the Cluster mission, the value is "STSP Cluster>Solar Terrestrial Science Programmes, Cluster". Other acceptable values are:
"LWS>Living With a Star""STP>Solar Terrestrial Probes""IACG>Inter-Agency Consultative Group""CDAWxx>Coordinated Data Analysis Workshop xx""SPDS>Space Physics Data System"
Others may be defined in the future. This attribute can have multiple entries if the data has been supplied to more than one project.
3.1.24. Rules_of_use¶
(Recommended.) Text containing information on, e.g., citability or use restrictions. This may point to a web page specifying the rules of use.
3.1.25. Skeleton_version¶
(Optional, required by Cluster.) Version of the skeleton file used to create the data file.
3.1.26. Software_version¶
(Optional, required by Cluster.) Version of the software that generated the data file.
3.1.27. Source_name¶
(Required.) This attribute identifies the mission or investigation that contains the sensors. For ISTP, this is the mission name for spacecraft missions or the investigation name for ground-based or theory investigations. Both a short name (used in Logical_source and in the filename) and a long name (used in Logical_source_description) separated by > are required. This attribute should be single valued. Examples:
"GEOTAIL>Geomagnetic Tail""WIND>Wind Interplanetary Plasma Laboratory""DARN>Dual Auroral Radar Network""GOES_7>Geostationary Operational Environmental Satellite 7""IMP-8>Interplanetary Monitoring Platform""LANL1989_046>Los Alamos National Laboratory 1989""C1>Cluster Satellite No 1"
3.1.28. spase_DatasetResourceID¶
(Recommended.) Unique dataset identifier assigned by a SPASE naming authority, of the form spase://NAMING_AUTHORITY/UNIQUE_ID, where UNIQUE_ID is the ID assigned by the NAMING_AUTHORITY. The SPASE resource record provides metadata about the dataset, including pointers to locations holding the data.
3.1.29. TEXT¶
(Required.) This attribute should contain description of the key measurements provided by the dataset. A short description of the experiment, with reference to a journal paper and/or web page describing the experiment in detail, is also essential. The attribute allows multiple entries. It is used by CDAWeb as the dataset description, and it is also included when data is requested as an ASCII listing.
3.1.30. Time_resolution¶
(Recommended.) Specifies time resolution of the data, e.g., "1 min"
3.1.31. TITLE¶
(Optional.) Dataset title, e.g., TITLE = "Parker Solar Probe ISOIS level 2 summary".
3.1.32. Validate¶
(Optional.) Details to be specified. This attribute is written by software for automatic validation of features such as the structure of the data file on a simple pass/fail criterion. The software will test that all expected attributes are present and, where possible, have reasonable values. The syntax is likely to be of the form "test>result>where done>date". It is not the same as data validation.
