From 5e0ee9254aa20bb84dc51ec69dd62a11c1569d54 Mon Sep 17 00:00:00 2001 From: Arno Kaimbacher Date: Tue, 18 Apr 2023 16:56:57 +0200 Subject: [PATCH] - add DataArchitectureDiagram.md --- DataArchitectureDiagram.md | 2 + Database.md | 12 +- data_diagram.drawio.svg | 613 +++++++++++++++++++++++++++++++++ tethys_acl.png:Zone.Identifier | 3 - 4 files changed, 625 insertions(+), 5 deletions(-) create mode 100644 DataArchitectureDiagram.md create mode 100644 data_diagram.drawio.svg delete mode 100644 tethys_acl.png:Zone.Identifier diff --git a/DataArchitectureDiagram.md b/DataArchitectureDiagram.md new file mode 100644 index 0000000..d0d0bf2 --- /dev/null +++ b/DataArchitectureDiagram.md @@ -0,0 +1,2 @@ +![Image](./data_diagram.drawio.svg) + diff --git a/Database.md b/Database.md index a69f835..8bcd452 100644 --- a/Database.md +++ b/Database.md @@ -106,6 +106,14 @@ In addition, the relation types (the kind of relationship) are precisely defined ## TETHYS lookup tables ![title](./tethys_lookup.png) -* **document_xml_cache**: +* **document_xml_cache**: all of the dataset's relations are stored in an internal XML format with a timestamp. Once the internal XML file has been created, it could be parsed and transformed into various output formats, including OAI-PMH (dublin core and datcite protocols used for OAI queries) and DOI registration data. These output formats would likely include information about the dataset itself (such as title, author, abstract, subjects, language, collection and publication date), as well as information about its relationships to other records in the repository. This metadata could then be used by other systems or services to facilitate discovery and access to the published dataset: + + * 'xml_data': stores the internal TETHYS xml. + * 'xml_version': if the database model changes, the version of the internal xml must be increased. + * 'server_date_modified': timestamp of dataset publication or doi update date * **mime_types**: -* **languages**: \ No newline at end of file + * 'name': this attribute holds the mime-type. e.g. 'text/csv', 'application/x-sqlite3', 'image/jpeg' + * 'file_extension': all allowed file extension for a mime-type: e.g. jpg|jpeg|jpe for the mime-type 'image/jpeg' + * 'enabled': defines if the mime-type is enabled for TETHYS + * timestamp attributes 'cretaed_at' and 'updated_at' +* **languages**: \ No newline at end of file diff --git a/data_diagram.drawio.svg b/data_diagram.drawio.svg new file mode 100644 index 0000000..400a82b --- /dev/null +++ b/data_diagram.drawio.svg @@ -0,0 +1,613 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+ Scanned Maps +
+
+
+ + + Scanned Maps + + + + + + +
+
+
+ Analog Data +
+
+
+ + + Analog Data + + + + + + + + + + + +
+
+
+ GIS Data +
+ (OGC Standards - Open Geospatial Consortium) +
+ Text Data +
+ (CSV, txt..) +
+
+
+
+ + + GIS Data... + + + + + + +
+
+
+ Digital Data +
+
+
+ + + Digital Data + + + + + + + + + + + +
+
+
+ XSL +
+
+
+ + + XSL + + + + + + + + +
+
+
+ TIFF +
+
+
+ + + TIFF + + + + + + + + +
+
+
+ PDF/A +
+
+
+ + + PDF/A + + + + + + + + +
+
+
+ GIS Data +
+
+
+ + + GIS Data + + + + + + +
+
+
+ Network Store +
+
+
+ + + Network Store + + + + + + + + +
+
+
+ ... +
+
+
+ + + ... + + + + + + + + +
+
+
+ Processing +
+ review +
+ commit +
+
+
+
+ + + Processing... + + + + + + +
+
+
+ Data Creation / curation +
+
+
+ + + Data Creation / curation + + + + + + + + + + + + +
+
+
+ + + Data Sources + + +
+
+
+ + + Data Sources + + + + + + + + + + + +
+
+
+ Data Integration +
+
+ + Storage Infrastructure + +
+
+
+
+ + + Data Integration... + + + + + + +
+
+
+ SOLR Search Index +
+
+
+ + + SOLR Search Index + + + + + + + +
+
+
+ Postgres/Postgis +
+ Database +
+
+
+ + + Postgres/Postgis... + + + + + + + + +
+
+
+ Postgres/Postgis +
+ Database +
+
+
+ + + Postgres/Postgis... + + + + + + + + + + + + + + +
+
+
+ Tethys (Ubuntu 22.04) +
+ NGINX +
+ JAVA 11 +
+ NodeJS +
+
+
+ + + Tethys (Ubuntu 22.04)... + + + + + + +
+
+
+ TETHYS Data Repository +
+
+
+ + + TETHYS Data Repo... + + + + + + +
+
+
+ Data Publication +
+
+
+ + + Data Publication + + + + + + + + + + + + +
+
+
+ Users +
+
+
+ + + Users + + + + + + + + + + + + +
+
+
+ + + Discovery and access systems + + +
+
+
+ + + Discovery and access systems + + + + + + +
+
+
+ + Frontend +
+ https://tethys.at/search + +
+
+
+ + + Frontend... + + + + + + +
+
+
+ + + REST API for metadata + +
+ + https://www.tethys.at/api/dataset/209 + +
+ +
+
+
+ + + REST API for metadata... + + + + + + + + +
+
+
+ + + DataCite Fabrica +
+ https://doi.datacite.org/dois/10.24341%2Ftethys.200 + +
+ +
+
+
+ + + DataCite Fabrica... + + + + + + + + +
+
+
+ + + OAI PMH + +
+ + https://tethys.at/oai?verb=Identify + +
+ +
+
+
+ + + OAI PMH... + + + + + + + + +
+
+
+ + SOLR Index +
+ https://tethys.at/search + +
+
+
+ + + SOLR Index... + + + + + + + + +
+
+
+ Access Control List (ACL) +
+
+
+ + + Access Control List (ACL) + + + + + + + + + Text is not SVG - cannot display + + + + \ No newline at end of file diff --git a/tethys_acl.png:Zone.Identifier b/tethys_acl.png:Zone.Identifier deleted file mode 100644 index 053d112..0000000 --- a/tethys_acl.png:Zone.Identifier +++ /dev/null @@ -1,3 +0,0 @@ -[ZoneTransfer] -ZoneId=3 -HostUrl=about:internet