Generating Outputs with R

You are here:
< All Topics
image_pdfDownload as PDF

aWhere Training Tutorial

Introduction

This tutorial aims to get you started with the main scripts that produce valuable data files (csv), graphs, and maps. This tutorial goes deeper into how to run the aWhere Training scripts that you downloaded in the Tutorial: Accessing aWhere’s R Scripts using Github and how to leverage these outputs. If it has been a few weeks since you downloaded the scripts we suggest making sure you have the most up-to-date version by downloading them once again from GitHub. This tutorial is going to start with the building blocks of aWhere’s R Scripts and then we will go through each of the training scripts in more detail. The first few steps will be the same for each script!

Building blocks of aWhere’s R Scripts:

There are a few lines of code that are the same in each script – these are what we often refer to as the parameters to set before running the code. The goal with the following few steps is to make sure you feel comfortable setting these parameters since they are the building blocks for the whole set of training scripts! In this section we will review how to setup the following parameters:

  • Loading the appropriate packages for each scripts
  • Loading your aWhere credentials (refer to Tutorial: Getting your aWhere Key and Secret if you do not have a credentials file yet!)
  • Setting your working directory (This is where the WorkProjects folder in the suggested file structure will be important!)
  • Loading your locations file (Remember those locations we collected in the Tutorial: Generating locations file using QGIS? This is where we will start digging deeper into those locations)!
  • Setting the time period for your data. Remember, aWhere’s data goes all the way to January 1, 2006 and includes the 15-day forecast! You can pull data for each location starting on this date and going forward 15 days.

Finding your pathname

All of aWhere’s scripts require you to specify where one or more files are located on your computer. This is done through the pathname. This section is often the most difficult for new R users but with a simple shortcut, there is no need to worry! 

There are a few ways you can find your pathname. You can type it directly (which is time consuming) or you can use the “copy as path” option on your computer. For this training, we will focus on using this shortcut.

If you are using a PC:

Open Windows Explorer and find the document you need to load.

Hold down the Shift key, then right-click the document.

In the context menu that appears, find and click Copy as path as seen in this image. This copies the file location to the clipboard. (Note: if you don’t hold down Shift when you right-click, the Copy as path option won’t appear.)

Navigate back to RStudio and Press Ctrl-V to paste in the file location in the appropriate line of code.

Once you confirm the pathname is correct, make sure to switch all of the back slashes ( \ ) to forward slashes ( / ) in your pathname

  • If using a Mac:
    • Navigate to the file or folder you wish to copy the path for. Right-click (or Control+Click, or a Two-Finger click on trackpads) on the file or folder in the Mac Finder. While in the right-click menu, hold down the OPTION key to reveal the “Copy (item name) as Pathname” option, it replaces the standard Copy option. 
    • You can then paste this pathname in the appropriate line of your code.
    • If using a Mac, your pathname should copy with forward slashes – this does not need to be changed.

The parameters that require pathnames include but are not limited to:

  • Working directory
  • Credentials file
  • Locations file
  • Supporting functions (where applicable)

In the next section, you will learn where to use this Copy as path shortcut as you work through the scripts.

Let’s get started: Setting parameters

Please load script 1-access_awhere_api_data.R into RStudio. We will review the steps for setting parameters section by section according to the list above:

Each aWhere script begins with an introduction
of the purpose of the script and any tips that might
be helpful as you move through the code.
Remember to read this code like you would any other
document! As mentioned previously, the lines that
are green and have a # in front are a comment and
are not lines you need to execute.
Installing and loading the appropriate packages
for each script:
For this section, you need to click “Run” for each line to
make sure you load the appropriate libraries.
Loading your aWhere credentials:
In order to access the API you need to load your credentials
file into R. When we say “load” that means that you essentially
need to specify where on your computer this file is saved
then R will go find it, read it, and load it!
For training purposes, we are going to use absolute
paths which includes the entire path name starting
from your base folder. Your base or root folder is likely
D, C, E, or F if you are a PC. For mac users, your
root folder is likely Users/yourName.

If you are using the suggested folder structure from the first
tutorial in this series you can type your pathfile according to
where your training folder is saved on your computer.
For example, if your main training folder is saved on your
root drive C your pathname for your credentials file might
look like the path below. 
Example pathname using “Copy as Path” shortcut:
C:/aWhere_training/Source/credentials.txt

Note: Your file pathname for your
credentials file will replace the
phrase”YOUR CREDENTIALS HERE” in
the above screenshot.
Setting your working directory:
If you are following the suggested aWhere training folder
structure, then your working directory will be your
WorkProjects folder. Note that you have an option to create
a subfolder within WorkProjects based on the analysis you
need to produce. For example, if you need to run the same
code every week for a month, perhaps you would like to
create subfolders for each week in order to keep your
outputs well organized. 

You will see a line of code in almost every script that starts
with dir.create(path = “outputCSVs/”. This helps further
organize your WorkProjects folder by putting the output
CSVs and the charts into separate folders.

Remember: your working directory is where all of the
charts, files, and maps created in R will be saved!
It is important you know where these folders are.  

Example pathname using “Copy as Path” shortcut: C:/aWhere_training/WorkProjects

Note: Your file pathname for your
working directory will replace the
phrase ”YOUR WD HERE” in the above screenshot.
Loading your locations file:
Did you know you can create outputs for 100’s or 1000’s
of locations with this script? Perhaps you want to monitor
the status of farmers you work with or understand where
weather variability is occurring based on your locations
file – these scripts will provide the outputs to make
data-driven decisions!

The locations file includes the coordinates (latitude, longitude) of the places you would like to investigate further. If you
created a locations file from the Tutorial: Generating a
locations file using QGIS then it should be saved in your RunSet folder. 
 
Example pathname using “Copy as Path” shortcut:
C:/aWhere_training/RunSet/locations.csv

OR it might look like this for scripts 2-5:


Note: Your file pathname for your locations
file will replace the phrase
”YOUR LOCATION FILE.txt” in the above
screenshot OR you paste over anything
between the quotations with your pathname (2nd image above).
Specifying the time period for your data:
The first image here shows the options for setting the time
period for the forecast in scripts #1-2 – which allows you to
pull data for the forecast up to 15 days.
In this image you will see lines that read:

day_start = as.character(Sys.Date())
day_end = as.character(Sys.Date()+7)

This line is using the date of your computer
(also known as System date or Sys. Date) as the starting
date and the ending date as +7 days from the date on
your machine. If you want to pull a 15-day forecast,
simply erase the number 7 and add 15.

For the second image, this is what you will find in
scripts #3-4. The two lines that designate the time
period of interest are:
date_start <- “2018-01-01”
date_end <- “2019-06-16”
This means that data will be pulled for every location
from Jan, 1, 2018 – June 16, 2019. aWhere’s data starts
in 2006 so you can theoretically pull data starting from
this time period. You can also go 15 days in to the
forecast based on the data needed. The most useful
time period is likely the rainy season OR we often pull an
entire year’s worth of data or create charts a year-long
period in order to understand what happened in those
locations last year or last rainy season.

Also, you can set the year_start or year_end for the
long-term normals (LTN) which are included in many
of the outputs. Perhaps you would like the LTN to include
only a certain 2 or 3-year period. You can set that using these lines:
Year_start <- 2006
Year_end <- 2018

OR it might look like this for scripts 3-4:

Note: Your time period of interest will be
inserted in between the quotation marks
above for the lines date_start and date_end.

Also, for script #4, you can identify additional years that you would like to add to the charts
created by that script. This is useful for looking
at particularly anomalous years or perhaps El Nino or La Nina years.

Supporting Functions:
For scripts 3-5, you will need to load additional code
located in the Supporting Functions file.
This file is located in your Source folder.
Example pathname using “Copy as Path” shortcut:
C:/aWhere_training/Source/Supporting_functions.R
Template file: SCRIPT 5 ONLY!!
Do you recall in the folder structure a file titled
“TemplateZambia.csv”? Now is the time we will use that
file in R. Script #5 is the only script that utilizes this file
and it creates powerful charts that include data compared
to population trends within your country of interest.
Example pathname using “Copy as Path” shortcut:
C:/aWhere_training/RunSet/TemplateZambia.csv
Weather data files from adaptER Platform:
SCRIPT 5 ONLY!!
Also utilized in script #5 are the weather data files you
downloaded in the Tutorial: Accessing Resources on the
adaptER Platform. These files can now be loaded into this
code to produce maps, statistics, and histograms.
You can load one or all of the 5 weather data files into
the code – it depends on your time period of interest and
whether or not you are looking at the forecast or past 30
days, or perhaps both.

Example pathname using “Copy as Path” shortcut:
C:/aWhere_training/BaseData/200219_next7.csv
Finishing up and running your code!
In aWhere’s scripts, once you have set your parameters
above by pointing R to the correct files in your training
folder you are ready to run the entire code.
When you come across a section title Beginning of Analysis
or in some scripts there will be a blue for that signifies a
loop that will automatically go through every location
in your file and create the outputs. This is a very quick
process and it requires no additional input from the user.
Using the Source button:
The Source button is very useful because it runs the
ENTIRE script at once and you do not have to run each
line one by one. This also ensures that you do not forget
to run a critical line – it happens! Once you have set your
parameters set you can click Source to run the entire code. 
This button is located to the right of Run. You will see the progress of your code in the Console pane.

Now, navigate to your WorkProjects folder to see your outputs. You can follow the above instructions to run all of the training scripts.

Review of all scripts and outputs

1-access_awhere_api_data

Purpose:

This code will show you how to access aWhere’s ag-weather datasets from the API (Application Programming Interface) for your location(s) of interest.

Prior to running this script, we encourage you to find the latitude and longitude of an area(s) of interest by using the QGIS tutorial on generating your locations file or by using GPS points that you have previously collected.

This script provides the following datasets for your location of interest:

  1. A csv output of the Forecast (Hourly, 6 hour, 12-hour, or daily blocks of time)
  2. Observed data for any time period between 2008 and present
  3. Long-Term Normals (LTN) for chosen time period between 2008 and present
  4. A csv output called the “aWhere Weather Dataset” which includes all observed variables and all LTN variables including the differences from normal.

Output type:

Spreadsheets saved as .csv files. After running script 1 with the sample location set for Zambia (created in previous tutorial), these are the .csv files that were created in the WorkProjects folder (outputCSVs subfolder which was created within the script).

Interpretation:

These raw data can be used for research, you can create your own tables in Excel, or you can use another software to analyze this data. This format is easily manipulated.

2-create_forecast_summary_table

Purpose:

This code will show you how to access aWhere’s forecast data from the API (Application Programming Interface) for your location(s) of interest to provide forecast data in a table and a csv file.

The outputs of this script include a csv of the forecast

(7-day, 168 hours) for your location of interest and a table of the forecast that can be embedded in reports or visuals for quick interpretation.

Output Type:

Forecast summary table:

Spreadsheet of forecast data saved as .csv file

Interpretation:

This forecast can be provided to farmers, field agents, and officials to better understand the weather patterns.

3-create_climatology_chart

Purpose:

This code will show you how to access aWhere’s API to create a chart that includes current maximum temperature, long-term maximum temperature, current precipitation, and long-term precipitation (plus standard deviation) for the time period and location(s) of interest that you specify.

Output Type:

The outputs of this script include a weekly climate chart for each location you added in your text file as well as a corresponding csv for further analysis (this is the raw data). The charts can be embedded in reports or visuals for quick interpretation.

Example chart:

Interpretation:

These charts are commonly used to understand the patterns over a year-long period. These charts help analysts (you!) visualize the precipitation per week along with the temperature patterns and how they compare to the long-term normal. In the chart above, the blue bars that surpass the orange line show greater than normal precipitation. You can start to ask questions such as: Is this enough rain to bring potential flooding? Is this rain in or out of season for rainfed crops?

4-create_charts

Purpose:

This code will show you how to quickly create a series of charts of your locations of interest to help you understand the weather/agronomic conditions in those locations based on aWhere’s ag-weather dataset (includes variables such as temperature, precipitation, potential evapotranspiration,

comparisons to long-term normals, and many more.) These charts can be added to reports for a quick analysis of the ag-weather conditions in your locations of interest. The weather data pulled for your location and time period of interest are also saved as a .csv file for future analysis.

Output Type:

The outputs of this script will create a series of 15 charts for each location in your locations file. The charts can be embedded in reports or visuals for quick interpretation. A subfolder for each location will be automatically created through the R code based on the name of the location. This provides a logical and organized folder hierarchy as shown below.

Interpretation:

The charts provided by this script allow the analyst to understand daily patterns of precipitation as well as rolling averages. Both are critical for rainfed agriculture and to understand if farmers are actually getting sufficient rain. These charts compare patterns to long-term normals and include advanced indices such as precipitation over potential evapotranspiration (P/PET). These concepts will be reviewed further in subsequent tutorials.

5-region_maps_stats_histograms

Purpose:

This script examines a region and returns maps, histograms and charts for the region and the sub-regions within it. It leverages data from the template file (including population data from WorldPop aggregated to aWhere’s 9km grid) as well as the weather data files to provide visuals of the current conditions. The section in the scripts titled “Region of Interest” allows you to define the region(s) within the country you want to investigate, also referred to as subsetting. This is based on the columns in the template file (admin0_name, admin1_name, etc). If you are not sure of the subregion names, please refer to the template file.

Output Type:

This script creates a series of maps and charts based on the regions you chose to subset as well as the number of weather data files you loaded. If you loaded all 5 weather files, you will have charts for each location PLUS each weather file – could amount to 100 charts!

This  script will also create subfolders for each region you chose to subset which keeps the charts well organized within your WorkProjects folder.

Tip: Notice how the WorkProjects folder now has multiple subfolders? These were created by running each script in this training session – all of the outputs will be well organized in these locations.

Interpretation:

These charts provide the current conditions for a country, district, county, or whichever subregion you chose to select.

What’s next? 

Keep practicing these scripts! You basically are learning a new “language” – R! Don’t be afraid to play around with different time periods and new locations files. Now that you have generated great charts, csvs, and figures, what should you do with them? We will review those concepts in the final Tutorial: Using Maps, Charts in Reporting. But first, we will be reviewing how to make a map layout using QGIS to accompany these charts in a professional report. Please open the Tutorial: Making a Map Layout in QGIS.

If you have any questions, please contact customersupport@awhere.com

Was this article helpful?
0 out Of 5 Stars
5 Stars 0%
4 Stars 0%
3 Stars 0%
2 Stars 0%
1 Stars 0%
How can we improve this article?
Table of Contents