geolba/tethys.backend
2
0
Fork 1
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

This page has exactly the same content as in Requirement 15 of the CTS documentation (reviewer's recommendation).

Porras-Bernardez 2023-11-29 08:13:43 +00:00
parent 0a6e6fc62d
commit 5767731115
1 changed files with 70 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

70
Technical-infrastructure.md Normal file

@ -0,0 +1,70 @@
The basic technical structure of Tethys RDR corresponds to a three-tiered client/server architecture with a number of clients and middleware components controlling the information flow and quality. On the server side, a RDBMS (PostgreSQL) is used for information storage (metadata, access control lists, file checksums). All metadata is replicated into middleware/frontend systems (Solr) for fast access and search capabilities. All public interfaces on the client side (tethys.frontend) to the information system are standards compliant (W3C, ISO, OGC) and are based on web services (REST API and SOLR search services). By using modern technologies like Bulma, Typescript, Vue, and Webpack, the repository's [frontend ](https://gitea.geologie.ac.at/geolba/tethys.frontend) is able to provide a more engaging and efficient user experience while also improving the overall performance. These technologies are widely used in the industry, so it is likely that the frontend will be compatible with a wide range of browsers and devices. The Tethys RDR REST Service, which is known as [tethys.api](https://gitea.geologie.ac.at/geolba/tethys.api), is built on top of the Express framework. Express is a popular web framework for Node.js that provides a minimalist set of features for deploying RESTful APIs that are easy to maintain and scale.
In order to comply with international metadata standards such as Dublin Core, DataCite and ISO 19139, the relevant information is delivered directly from the database by mapping (XSLT Transformation) [on the fly](https://www.tethys.at/oai?verb=ListMetadataFormats). By mapping Tethys RDR metadata to these international standards, it becomes easier for other systems to exchange and integrate metadata, which in turn can facilitate the discovery, access, and reuse of data and other resources (e.g. OGC CSW-Metadata Harvesting (ISO19139) from the repository's OAI-PMH base [URL](https://www.tethys.at/oai?)).
The new editorial system of the repository on the server side, which is also known as [tethys.backend](https://gitea.geologie.ac.at/geolba/tethys.backend), is a web-based open-source software that operates directly on the PostgreSQL database. It is built using AdonisJS, which is a Node.js-based web framework that provides a robust set of tools and features for building scalable and secure web applications. AdonisJS also includes an object-relational mapper (ORM) that allows developers to work with database tables and records using object-oriented programming techniques. The styling of the Tethys RDR backend is built using Tailwind, which is a utility-first CSS framework.
General descriptions of the systems and software used can be found in our Wiki [6]. There, it is possible to find public information about the database model, instructions for starting Docker container and a data architecture diagram for a clear understanding of all storage locations. All code repositories are accessible online via a Gitea instance [7] hosted at the GeoSphere Austria data centre. The associated Tethys RDR Docker images are also securely stored there using the built-in Docker registry functionality [8]. All necessary configurations to launch the Docker container are described in the Wiki.
Internal information about virtual server, maintenance and security settings are stored in a separate, private wiki on internal LAN servers.
Hardware infrastructure is generally renewed every 3-4 years, which is transparent to the system because of virtualization. Operating systems are regularly updated to the latest releases and patches.
GeoSphere Austria acts according to international IT standards for information security management. Compliance with these standards is monitored by the organization and ongoing audits are carried out as an ongoing process to ensure relevant standards are met.
All the software components used for the three-tiered client/server architecture in the Tethys RDR are versioned through GIT and can be accessed online via a Gitea instance [9] hosted by the computer centre of GeoSphere Austria. The frontend can be accessed at [10], the backend at [11], and the API at [12].
The Tethys RDR is being developed using the Continuous Integration/Continuous Deployment (CI/CD) practice, which involves frequent testing and deployment of code changes to production. Whenever new code is published, automatic tests run in the background (CI), and when new versions are released, new Docker images are automatically deployed (CD). This entire process is implemented using Gitea Actions, a powerful platform that enables automation of various tasks like building, testing, and deploying code, as well as sending notifications such as alerts for failed tests. All workflows are defined in YAML files, such as the ci.yaml file [13], which is always triggered whenever new code is committed to the repository's backend.
---
To ensure that the availability, bandwidth, and connectivity are sufficient to meet the needs of the Designated Community of the repository, the following measures are taken:
**1. A reliable and reputable hosting provider:**
All basic hard- and software services of Tethys RDR are hosted by the Geosphere Austria computer centre, which is a reliable provider that supplies the necessary infrastructure and software services.
All the necessary backend and middleware systems for the repository, as well as the web servers and SOLR search server for the frontend, are operating on virtual VMware servers. This setup provides sufficient capacity and high performance, as well as enhanced availability through virtualization. The dedicated machine responsible for the PostgreSQL database is equipped with a powerful IO system to ensure high performance. The server operating systems responsible for the Tethys RDR frontend and backend are operating on virtualized Ubuntu.
The use of professional monitoring software like ICINGA helps to ensure that the repository is running smoothly and that its users can access the data they need without interruption. The Tethys RDR utilizes the open-source web analytics software, Matomo, to monitor access statistics and gain insights into user behaviour.
**2. Caching mechanisms:**
The Tethys RDR REST API has implemented caching mechanisms, which are based on REDIS cache. REDIS provides fast in-memory data storage and retrieval to reduce the load on the repository's servers and speed up access to frequently accessed content.
**3. Load balancing:**
The repository uses NGINX as a load balancer to distribute traffic across multiple servers to ensure that it can handle high levels of traffic and provide a seamless user experience.
**4. Performance monitoring and capacity planning:**
The Tethys RDR staff regularly monitors the performance of the repository and performs disk capacity planning to ensure that the repository can handle future growth in usage.
**5. Multiple access points:**
The repository provides multiple access points, including web interfaces, REST APIs, and web applications. The frontend web application employs a full responsive frontend design to ensure that users can access the data from a variety of devices and platforms. This means that the repository's web interface is optimized to provide an optimal viewing and interaction experience across a wide range of screen sizes and device types, including desktops, laptops, tablets, and smartphones.
There are several processes that are used to monitor and manage the need of technical changes in software development for the Tethys RDR.
- Change management: This involves establishing a formal process for requesting, reviewing, approving, and implementing changes to software development. Changes may include new features, bug fixes, or modifications to existing functionality.
- Version control: This enables the developer team to keep track of changes to software code and ensure that everyone is working on the most up-to-date version. This is done using Git.
- Continuous integration/continuous delivery (CI/CD): These are practices that involve automating the building, testing, and deployment of software. This helps to catch errors and ensure that changes are released into production environments as quickly and reliably as possible.
- Agile methodologies: Agile development focuses on iterative development cycles, with frequent feedback and collaboration between developers, stakeholders, and end-users. This approach helps ensure that software development stays aligned with changing requirements and priorities.
---
References:
[1] https://gitea.geologie.ac.at/geolba/tethys.frontend
[2] https://gitea.geologie.ac.at/geolba/tethys.api
[3] https://www.tethys.at/oai?verb=ListMetadataFormats
[4] https://www.tethys.at/oai?
[5] https://gitea.geologie.ac.at/geolba/tethys.backend
[6] https://gitea.geologie.ac.at/geolba/tethys.backend/wiki/?action=_pages
[7] https://gitea.geologie.ac.at
[8] https://gitea.geologie.ac.at/geolba/-/packages
[9] https://gitea.geologie.ac.at
[10] https://gitea.geologie.ac.at/geolba/tethys.frontend
[11] https://gitea.geologie.ac.at/geolba/tethys.backend
[12] https://gitea.geologie.ac.at/geolba/tethys.api
[13] https://gitea.geologie.ac.at/geolba/tethys.backend/actions/runs/5