README.md

# C3S-512 CDS Data Checker


## Install & Run

```bash
conda create -y -n dqc python=3
conda activate dqc
git clone https://earth.bsc.es/gitlab/external/c3s512-wp1-datachecker.git
cd c3s512-wp1-datachecker
pip install -r requirements.txt
cd dqc_chekcer

python checker.py <config_file>

```
**Note**: In the following section you will find information on how to write your own **config_file**.

## Configure

```bash
- In order to run the checker you must write a simple config
- There is a general section where general dataset and path options are specified
- Each config section represents a check (ex: file_format and temporal_completeness)
- Each config section might have specific parameters related to the specific check

```
**Note 1**: Config examples for **ALL** available checks can be found in the **dqc_wrapper/conf** folder.
**Note 2**: The following config checks for temporal consistency. Multiple checks can be stacked in one file.

````
[general]
input = /path/to/folder/or/file
log_dir = /path/to/log_folder
type = netcdf

[temporal_completeness]
time_start = YYYY-MM-DD
time_end = YYYY-MM-DD
````

## Recent updates

You can find an updated LOG to track new major modifications here:<br>
* [UPDATE LOG](UPDATE_LOG.md) 

## Description

The main function of this Gitlab Project is to join all the efforts done in the evaluation of the **C**limate **D**ata **S**tore (**CDS**).<br><br>
You can find these things:
* [Summary of Avalable Data Checkers](01_summary_data_checkers.md)
* [Fist dataset inventory of the CDS](02_cds_inventory.md)
* [Download and Format LOGs for Files from the CDS](LOG_download_format/download_and_format_info.md)
* [Download scripts used for the first assesment](download_scripts/)
* [First CF check LOG using existing cfchecker for NetCDF files](CF_checker_log/)
* [CDS API Scrapping for metadata and variables](cds_metadata/)
* [Grib Files Checker](grib_checker)
<br><br>