Read ECMWF data ERA 5

ERA5 is the latest development in the ERA series, and  improves significantly on its predecessors by:

- Offering a higher horizontal resolution of 31 km and 137 vertical levels from the surface up to 0.01 hPa (around 80 km);

- Using a more recent and advanced version of the ECMWF IFS model;

- Providing hourly estimates of atmospheric variables;

- Providing a consistent representation of uncertainties for these variables;

- Using more satellite observations in the data assimilation.

Here are some information to access this database using python batch

Please visit http://climate.copernicus.eu/climate-reanalysis for more information including documentation and guidelines on how to download the data.

https://codes.ecmwf.int/grib/param-db/139

Layer 1: 0 - 7cm

Layer 2: 7 - 28cm

Layer 3: 28 - 100cm

Layer 4: 100 - 289cm

Soil temperature is set at the middle of each layer, and heat transfer is calculated at the interfaces between them. It is assumed that there is no heat transfer out of the bottom of the lowest layer.

__________

June 2025

The new ECMWF methodology cannot use R directly. The solution I found to download data in MacOSX was:

https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels?tab=overview

To install the CDSAPI using pip, the solution is to create a virtual environnement of python:

For example, here the virtual environment is py_envs:

python3 -m venv ~/py_envs

After the environment is created, you need doing only:

source ~/py_envs/bin/activate

# Within this environment, I can install the cdsapi using pip:

pip install cdsapi

# And then I can run the python code downloaded from https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels?tab=overview

For example here the code is store in loadFl:

cat loadFl

_______________________

import cdsapi
dataset = "reanalysis-era5-single-levels"
request = {
    "product_type": ["reanalysis"],
    "variable": ["2m_temperature"],
    "year": [
        "2019", "2020", "2021",
        "2022", "2023", "2024"
    ],
    "month": [
        "01", "02", "03",
        "04", "05", "06",
        "07", "08", "09",
        "10", "11", "12"
    ],
    "day": [
        "01", "02", "03",
        "04", "05", "06",
        "07", "08", "09",
        "10", "11", "12",
        "13", "14", "15",
        "16", "17", "18",
        "19", "20", "21",
        "22", "23", "24",
        "25", "26", "27",
        "28", "29", "30",
        "31"
    ],
    "time": [
        "00:00", "01:00", "02:00",
        "03:00", "04:00", "05:00",
        "06:00", "07:00", "08:00",
        "09:00", "10:00", "11:00",
        "12:00", "13:00", "14:00",
        "15:00", "16:00", "17:00",
        "18:00", "19:00", "20:00",
        "21:00", "22:00", "23:00"
    ],
    "data_format": "grib",
    "download_format": "zip",
    "area": [26.5, -80.25, 26.25, -80]
}
client = cdsapi.Client()
client.retrieve(dataset, request).download()

The last line can be:
client.retrieve(dataset, request, filename)
To download directly the file with a filename; 
Note that R package ncdf4 cannot read this later file.

____________

It can be run using:

python3 loadFl

Then go to:

https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels?tab=overview

Follow the link "Your request"

And you can monitor your request and download the result when it is done.




Commentaires

Enregistrer un commentaire

Posts les plus consultés de ce blog

Standard error from Hessian Matrix... what can be done when problem occurs

An Hessian matrix is a square matrix of partial second order derivative. From the square-root of the inverse of the diagonal, it is possible to estimate the standard error of parameters. Let take an example: val=rnorm(100, mean=20, sd=5) # Return -ln L of values in val in Gaussian distribution with mean and sd in par fitnorm<-function(par, val) {   -sum(dnorm(val, par["mean"], par["sd"], log = TRUE)) } # Initial values for search p<-c(mean=20, sd=5) # fit the model result <- optim(par=p, fn=fitnorm, val=val, method="BFGS", hessian=TRUE) # Inverse the hessian matrix to get SE for each parameters mathessian <- result$hessian inversemathessian <- solve(mathessian) res <- sqrt(diag(inversemathessian)) # results data.frame(Mean=result$par, SE=res)   library(MASS) (r<-fitdistr(val, "normal")) It works well. However, the inverse of the matrix cannot be calculated if the second order derivative for some paramete...
Lire la suite

Multivariable analysis and correlation of iconography

Image
Introduction Correlation iconography is a not very well known method to study multivariate data. It was developed long-time ago by Michel Lesty: Lesty M (1999) Une nouvelle approche dans le choix des régresseurs de la régression multiple en présence d'intéractions et de colinearités. La revue de Modulad 22:41-77 It is also well described in a French Wikipedia page: https://fr.wikipedia.org/wiki/Iconographie_des_corrélations After checking the possibilities of this method, I think that it deserves more attention. Let take the example of the wikipedia page: dta <- read.table(text=gsub(",", ".", "Élève Poids Âge Assiduité Note e1 52 12 12 5 e2 59 12,5 9 5 e3 55 13 15 9 e4 58 14,5 5 5 e5 66 15,5 11 13,5 e6 62 16 15 18 e7 63 17 12 18 e8 69 18 9 18"), header=TRUE) > dta   Élève Poids  Âge Assiduité Note 1    e1    52 12.0        12  5.0 2    e...
Lire la suite

Install treemix in ubuntu 20.04

Informations are available here  https://bitbucket.org/nygcresearch/treemix/downloads/ In terminal, enter for  Ubuntu 20.04: wget https://bitbucket.org/nygcresearch/treemix/downloads/treemix-1.13.tar.gz tar -xvf treemix-1.13.tar.gz sudo apt-get install libgsl-dev sudo apt install libboost-dev sudo apt install libboost-all-dev cd treemix-1.13/ ./configure make sudo make install And it works: $ treemix TreeMix v. 1.13 $Revision: 231 $ Options: -h display this help -i [file name] input file -o [stem] output stem (will be [stem].treeout.gz, [stem].cov.gz, [stem].modelcov.gz) -k [int] number of SNPs per block for estimation of covariance matrix (1) -global Do a round of global rearrangements after adding all populations -tf [file name] Read the tree topology from a file, rather than estimating it -m [int] number of migration edges to add (0) -root [string] comma-delimited list of populations to set on one side of the root (for migration) -g [vertices file name] [edges file na...
Lire la suite