Get the required parameters from the Met Office Unified Model (TM)
(hereafter MetUM) to use in FVCOM surface forcing.
met_files = get_MetUM_pp(modelTime, credentials)
DESCRIPTION:
Using FTP access, extract the necessary parameters to create an FVCOM
forcing file. Requires the air_sea toolbox (see below for where to get
it). Data are sampled four times daily.
INPUT:
modelTime - Modified Julian Date start and end times array
credentials - cell array of username and password to access the BADC
FTP server.
OUTPUT:
met_files - cell array of file names downloaded from the BADC servers.
The PP files downloaded give:
- surface_net_downward_shortwave_flux (W m-2)
- surface_downwelling_shortwave_flux_in_air (W m-2)
- surface_net_downward_longwave_flux (W m-2)
- surface_downwelling_longwave_flux_in_air (W m-2)
- surface_upward_latent_heat_flux (W m-2)
- surface_upward_sensible_heat_flux (W m-2)
- eastward_wind / x_wind (m s-1)
- northward_wind / y_wind (m s-1)
- surface_upward_latent_heat_flux (W m-2)
- air_temperature (K)
- relative_humidity (%)
- precipitation_flux (kg m-2 s-1)
- air_pressure_at_sea_level (Pa)
In addition to these, the momentum flux is calculated from wind data.
Precipitation is converted from kg/m^2/s to m/s. Evaporation is
calculated from the mean daily latent heat net flux (lhtfl) at the
surface.
EXAMPLE USAGE:
met_files = get_MetUM_pp([51725, 51757], {'username', 'password'});
TODO:
Add support for the AP directories on the FTP server.
Author(s)
Pierre Cazenave (Plymouth Marine Laboratory)
Revision history:
2013-05-07 First version.
2013-06-24 Update some of the code to correctly separate the different
directories (e.g. am vs mn in 2010). Also farm out the FTP and
conversion from PP format to NetCDF to separate functions
(get_BADC_data.m and pp2nc.m, respectively). Renamed the function from
get_MetUM_forcing to get_MetUM_pp to better reflect what it does.
2013-09-11 Add support for the post-2011 output files. Also make the
downloads in parallel (this does abuse the BADC somewhat if you have
loads of threads running and will still only go as fast as they
throttle individual connections, assuming they do that).
==========================================================================0001 function met_files = get_MetUM_pp(modelTime, credentials) 0002 % Get the required parameters from the Met Office Unified Model (TM) 0003 % (hereafter MetUM) to use in FVCOM surface forcing. 0004 % 0005 % met_files = get_MetUM_pp(modelTime, credentials) 0006 % 0007 % DESCRIPTION: 0008 % Using FTP access, extract the necessary parameters to create an FVCOM 0009 % forcing file. Requires the air_sea toolbox (see below for where to get 0010 % it). Data are sampled four times daily. 0011 % 0012 % INPUT: 0013 % modelTime - Modified Julian Date start and end times array 0014 % credentials - cell array of username and password to access the BADC 0015 % FTP server. 0016 % 0017 % OUTPUT: 0018 % met_files - cell array of file names downloaded from the BADC servers. 0019 % 0020 % The PP files downloaded give: 0021 % - surface_net_downward_shortwave_flux (W m-2) 0022 % - surface_downwelling_shortwave_flux_in_air (W m-2) 0023 % - surface_net_downward_longwave_flux (W m-2) 0024 % - surface_downwelling_longwave_flux_in_air (W m-2) 0025 % - surface_upward_latent_heat_flux (W m-2) 0026 % - surface_upward_sensible_heat_flux (W m-2) 0027 % - eastward_wind / x_wind (m s-1) 0028 % - northward_wind / y_wind (m s-1) 0029 % - surface_upward_latent_heat_flux (W m-2) 0030 % - air_temperature (K) 0031 % - relative_humidity (%) 0032 % - precipitation_flux (kg m-2 s-1) 0033 % - air_pressure_at_sea_level (Pa) 0034 % 0035 % In addition to these, the momentum flux is calculated from wind data. 0036 % Precipitation is converted from kg/m^2/s to m/s. Evaporation is 0037 % calculated from the mean daily latent heat net flux (lhtfl) at the 0038 % surface. 0039 % 0040 % EXAMPLE USAGE: 0041 % met_files = get_MetUM_pp([51725, 51757], {'username', 'password'}); 0042 % 0043 % TODO: 0044 % Add support for the AP directories on the FTP server. 0045 % 0046 % Author(s) 0047 % Pierre Cazenave (Plymouth Marine Laboratory) 0048 % 0049 % Revision history: 0050 % 2013-05-07 First version. 0051 % 2013-06-24 Update some of the code to correctly separate the different 0052 % directories (e.g. am vs mn in 2010). Also farm out the FTP and 0053 % conversion from PP format to NetCDF to separate functions 0054 % (get_BADC_data.m and pp2nc.m, respectively). Renamed the function from 0055 % get_MetUM_forcing to get_MetUM_pp to better reflect what it does. 0056 % 2013-09-11 Add support for the post-2011 output files. Also make the 0057 % downloads in parallel (this does abuse the BADC somewhat if you have 0058 % loads of threads running and will still only go as fast as they 0059 % throttle individual connections, assuming they do that). 0060 % 0061 %========================================================================== 0062 0063 subname = 'get_MetUM_forcing'; 0064 0065 global ftbverbose 0066 if ftbverbose 0067 fprintf('\nbegin : %s \n', subname) 0068 end 0069 0070 % Run jobs on multiple workers if we have that functionality. Not sure if 0071 % it's necessary, but check we have the Parallel Toolbox first. 0072 wasOpened = false; 0073 if license('test', 'Distrib_Computing_Toolbox') 0074 % We have the Parallel Computing Toolbox, so launch a bunch of workers. 0075 if matlabpool('size') == 0 0076 % Force pool to be local in case we have remote pools available. 0077 matlabpool open local 0078 wasOpened = true; 0079 end 0080 end 0081 0082 nt = ceil(modelTime(end)) - floor(modelTime(1)); 0083 if nt > 365 0084 error('Can''t (yet) process more than a year at a time.') 0085 end 0086 0087 yearStart = mjulian2greg(modelTime(1)); 0088 assert(yearStart >= 2006, 'The MetUM repository does not contain data earlier than 2006.') 0089 0090 % Four times daily outputs at 0000, 0600, 1200 and 1800 0091 t = modelTime(1):1/4:modelTime(end); 0092 0093 % For the pre-2010 data, we need to download several files with unique 0094 % names. The names are based on the STASH numbers and the date: 0095 % naamYYYYMMDDHH_STASH#_00.pp 0096 % The numbers we're interested in are stored in stash. 0097 stash = [2, 3, 407, 408, 409, 4222, 9229, 16004, ... 0098 1201, 1235, 2207, 2201, 3217, 3225, 3226, 3234, 3236, 3237, 3245, ... 0099 5216, 16222, 20004]; 0100 % The stash numbers and their corresponding forcing type. 0101 % 0102 % AP = analysis, pressure levels 0103 % AM = analysis, model levels 0104 % 0105 % |---------|-------------------------------------------| 0106 % | stash # | forcing type | 0107 % |-----------------------------------------------------| 0108 % | AM | 0109 % |-----------------------------------------------------| 0110 % | 2 | eastward_wind / x_wind | 0111 % | 3 | northward_wind / y_wind | 0112 % | 407 | air_pressure | 0113 % | 408 | air_pressure | 0114 % | 409 | surface_air_pressure | 0115 % | 4222 | RAILFALL RATE OUT OF MODEL LEVELS | 0116 % | 9229 | RELATIVE HUMIDITY AFTER MAIN CLOUD | 0117 % | 16004 | air_temperature | 0118 % |-----------------------------------------------------| 0119 % | AP | 0120 % |-----------------------------------------------------| 0121 % | 1201 | surface_net_downward_shortwave_flux | 0122 % | 1235 | surface_downwelling_shortwave_flux_in_air | 0123 % | 2207 | surface_downwelling_longwave_flux_in_air | 0124 % | 2201 | surface_net_downward_longwave_flux | 0125 % | 3217 | surface_upward_sensible_heat_flux | 0126 % | 3225 | eastward_wind / x_wind | 0127 % | 3226 | northward_wind / y_wind | 0128 % | 3234 | surface_upward_latent_heat_flux | 0129 % | 3236 | air_temperature | 0130 % | 3237 | specific_humidity | 0131 % | 3245 | relative_humidity | 0132 % | 5216 | precipitation_flux | 0133 % | 16222 | air_pressure_at_sea_level | 0134 % | 20004 | [RIVER OUTFLOW] | 0135 % |---------|-------------------------------------------| 0136 % 0137 % For the post-2010 data, the stash codes are depracated in favour of 0138 % single files with individual variables. As such, in those instances, the 0139 % four data files are downloaded and then the extraction of the variables 0140 % of interest can be done when the PP files have been converted to netCDF 0141 % and are subsequently read in with read_MetUM_forcing.m. 0142 0143 ns = length(stash); 0144 0145 % From where will we be downloading the data? 0146 site = 'ftp.ceda.ac.uk'; 0147 0148 % Preallocate the output cell arrays. 0149 tmp_met_files = cell(1, nt * 4); 0150 met_files = cell(1, nt * 4); 0151 0152 % Depending on the year we're extracting, we need to append different 0153 % directories to get the data. 0154 parfor i = 1:nt * 4 % four files per day (at 0000, 0600, 1200 and 1800). 0155 0156 [year, month, day, hour] = mjulian2greg(t(i)); 0157 0158 % Pre-2012 data are in a different directory from the post-2012 data, 0159 % so adjust accordingly here. 0160 if year < 2012 0161 basePath = 'badc/ukmo-um/data/nae/'; 0162 else 0163 basePath = 'badc/ukmo-nwp/data/nae/all_years'; 0164 end 0165 0166 % Cell array for the files to download. 0167 files = cell(0); 0168 0169 % Do 2010 first because it straddles the two directories. 0170 if year == 2010 0171 % Use modified julian dates for the thresholds for each directory. 0172 amthresh = greg2mjulian(2010, 11, 03, 00, 00, 00); 0173 0174 if greg2mjulian(year, month, day, hour, 00, 00) <= amthresh 0175 % Use the am data 0176 prefix = 'am'; 0177 filepath = sprintf('%sna/%s/%04d/%02d/%02d', basePath, ... 0178 prefix, ... 0179 year, ... 0180 month, ... 0181 day); 0182 for f = 1:ns 0183 files{f} = sprintf('na%s%04d%02d%02d%02d_%05d_00.pp', ... 0184 prefix, ... 0185 year, ... 0186 month, ... 0187 day, ... 0188 hour, ... 0189 stash(f)); 0190 end 0191 else 0192 % Use the mn data 0193 prefix = 'sn'; 0194 filepath = sprintf('%sna/%s/%04d/%02d/%02d', basePath, ... 0195 prefix, ... 0196 year, ... 0197 month, ... 0198 day); 0199 files{length(files + 1)} = sprintf('%s_%04d%02d%02d%02d_s00.pp', ... 0200 prefix, ... 0201 year, ... 0202 month, ... 0203 day, ... 0204 hour); 0205 end 0206 sprintf('%s', filepath); 0207 0208 % Check the 2006 data are from the 7th November onwards. 0209 elseif year == 2006 0210 if month < 11 0211 if day < 7 0212 error('The MetUM repository does not contain data earlier than 7th November, 2006') 0213 else 0214 prefix = 'am'; 0215 filepath = sprintf('%sna/%s/%04d/%02d/%02d', basePath, ... 0216 prefix, ... 0217 year, ... 0218 month, ... 0219 day); 0220 for f = 1:ns 0221 files{f} = sprintf('na%s%04d%02d%02d%02d_%05d_00.pp', ... 0222 prefix, ... 0223 year, ... 0224 month, ... 0225 day, ... 0226 hour, ... 0227 stash(f)); 0228 end 0229 end 0230 end 0231 0232 % Pre-2010 files. 0233 elseif year < 2010 0234 % Use the am data. 0235 prefix = 'am'; 0236 filepath = sprintf('%sna/%s/%04d/%02d/%02d', basePath, ... 0237 prefix, ... 0238 year, ... 0239 month, ... 0240 day); 0241 for f = 1:ns 0242 files{f} = sprintf('na%s%04d%02d%02d%02d_%05d_00.pp', ... 0243 prefix, ... 0244 year, ... 0245 month, ... 0246 day, ... 0247 hour, ... 0248 stash(f)); 0249 end 0250 0251 % 2011-2012 files. 0252 elseif year > 2010 && year < 2012 0253 % Use the sn data (has everything we need but doesn't have 70 0254 % vertical levels!). 0255 prefix = 'sn'; 0256 filepath = sprintf('%sna/%s/%04d/%02d/%02d', basePath, ... 0257 prefix, ... 0258 year, ... 0259 month, ... 0260 day); 0261 files{length(files) + 1} = sprintf('%s_%04d%02d%02d%02d_s00.pp', ... 0262 prefix, ... 0263 year, ... 0264 month, ... 0265 day, ... 0266 hour); 0267 0268 elseif year >= 2012 0269 % Marginally more straightforward in this case. 0270 prefix = 'prods_op_nae-mn'; 0271 filepath = sprintf('%s/%02d/%02d/%02d', basePath, year, month, day); 0272 % prods_op_nae-mn_20120101_00_012.pp 0273 files{length(files) + 1} = sprintf('%s_%04d%02d%02d_%02d_012.pp', ... 0274 prefix, ... 0275 year, ... 0276 month, ... 0277 day, ... 0278 hour); 0279 end 0280 0281 tmp_met_files{i} = get_BADC_data(site, filepath, files, credentials); 0282 0283 end 0284 0285 % Clean up the output to be a single cell array of file names. 0286 c = 0; 0287 for i = 1:length(tmp_met_files) 0288 for j = 1:length(tmp_met_files{i}) 0289 c = c + 1; 0290 met_files(c) = tmp_met_files{i}{j}; 0291 end 0292 end 0293 0294 % Close the MATLAB pool if we opened it. 0295 if wasOpened 0296 matlabpool close 0297 end 0298 0299 if ftbverbose 0300 fprintf('end : %s \n', subname) 0301 end