{"id":3059,"date":"2018-07-10T14:19:39","date_gmt":"2018-07-10T14:19:39","guid":{"rendered":"https:\/\/ermlab.com\/?p=3059"},"modified":"2018-09-12T20:52:41","modified_gmt":"2018-09-12T20:52:41","slug":"pandas-seaborn-world-bank-gdp-analysis","status":"publish","type":"post","link":"https:\/\/ermlab.com\/en\/blog\/data-science\/pandas-seaborn-world-bank-gdp-analysis\/","title":{"rendered":"The World Bank GDP Analysis using Pandas and Seaborn Python libraries"},"content":{"rendered":"

Pandas<\/span><\/a>\u00a0and\u00a0<\/span>Seaborn<\/span><\/a>\u00a0are one of the most useful data science related Python libraries. The first one provides an easy to use and high-performance data structures and methods\u00a0for data manipulation. The latter\u00a0is build\u00a0on top of matplotlib and provides a high-level interface for drawing attractive statistical graphics. How do they work?<\/span><\/p>\n

Let\u2019s check it out using World Bank GDP data from 10 central European countries – Poland, Germany, Belarus, the Czech Republic, the Slovak Republic, Hungary, Estonia, France, Ukraine\u00a0and\u00a0the United Kingdom.<\/span><\/p>\n

<\/p>\n

What are we looking for?<\/span><\/h2>\n

The question – How far in economic\u00a0development\u00a0eastern Europe countries are relative to developed countries like Germany and France?<\/span><\/p>\n

To answer it we need to analyze four GDP factors – GDP per capita (US$),\u00a0GDO\u00a0per capita growth (annual %), GDP growth (annual %) and GDP (current US$).<\/p>\n

The data from the World Bank (from the\u00a0<\/span>World Development Indicators<\/span><\/a>\u00a0website to be exact) are in an open format and have good\u00a0history\u00a0records for many countries that include a number of economic and social indicators.<\/span><\/p>\n

We chose the years 1990 – 2016 because only these were available for the selected indicators.<\/p>\n

You can find the data\u00a0here<\/a>.<\/p>\n

The code<\/span><\/h2>\n

First, we load the data from the CSV\u00a0file. Then we remove the last 5\u00a0lines,\u00a0because they contain empty values and information about the date of the last data update. In addition, we have to remove the column with the year 2016, because, as it turned out it is empty (no data). “gdp.replace<\/em>” is responsible for the replacement of two dots, symbolizing the empty NaN.<\/p>\n

import numpy as np\r\nimport pandas as pd\r\nimport matplotlib as mpl\r\nimport matplotlib.pyplot as plt\r\n \r\ngdp = pd.read_csv('.\/shared\/WorldBank\/GDP_Poland_neighbours.csv')\r\n \r\n#we take only data, not additional informations\r\ngdp = gdp[0:-5]\r\n#delete empty column\r\ndel gdp['2016 [YR2016]']\r\n#replace '..' string with nan values\r\ngdp.replace('..', np.nan, inplace=True)<\/pre>\n
In the course of further work with DataFrame I received mysterious errors and at first, I was not able to determine what was wrong. After some time I decided to check the types of the individual columns:<\/p>\n
gdp.dtypes\r\n \r\nCountry Name      object\r\nCountry Code      object\r\nSeries Name       object\r\nSeries Code       object\r\n1990 [YR1990]     object\r\n1991 [YR1991]     object\r\n1992 [YR1992]     object\r\n1993 [YR1993]     object\r\n1994 [YR1994]     object\r\n1995 [YR1995]     object\r\n1996 [YR1996]    float64\r\n1997 [YR1997]    float64\r\n1998 [YR1998]    float64\r\n1999 [YR1999]    float64\r\n2000 [YR2000]    float64\r\n2001 [YR2001]    float64\r\n2002 [YR2002]    float64\r\n2003 [YR2003]    float64\r\n2004 [YR2004]    float64\r\n2005 [YR2005]    float64\r\n2006 [YR2006]    float64\r\n2007 [YR2007]    float64\r\n2008 [YR2008]    float64\r\n2009 [YR2009]    float64\r\n2010 [YR2010]    float64\r\n2011 [YR2011]    float64\r\n2012 [YR2012]    float64\r\n2013 [YR2013]    float64\r\n2014 [YR2014]    float64\r\n2015 [YR2015]    float64\r\ndtype: objec<\/pre>\n
To my surprise dates from 1990 to 1995 didn’t have the data type float64\u00a0only\u00a0object, so I decided to be sure all the columns of years to convert to numeric values. For this purpose, I\u00a0select \u00a0columns\u00a0from 4 up to the end (that is, all of the years) and with use of “apply<\/em>” method ‘I applied the function “pd.to_numeric<\/em>“. It converts all years to a floating point number.<\/p>\n
# some of the colums are objects, we have to convert to floats, \r\n#then pivot_table will take them into consideration\r\ncol_list = gdp.columns[4:].values\r\ngdp[col_list]=gdp[col_list].apply(pd.to_numeric)<\/pre>\n
In each row, was the name of the country, its code, the name of a series of data from the World Bank, its code, and in subsequent columns the years. Such\u00a0arrangement\u00a0of the data was not too comfortable so I decided to reindex the table using the functions “pivot_table<\/em>”<\/p>\n
#reindex all table, create pivot view\r\npv2 = pd.pivot_table(gdp,index=['Series Name','Country Code'], dropna=False, fill_value=0.0)\r\n# set the years\r\npv2.columns= np.arange(1990,2016)<\/pre>\n
This has changed\u00a0dataframe\u00a0from\u00a0form:<\/p>\n
\"worldbank<\/p>\n
To this one.<\/p>\n
\"worldbank<\/p>\n
That way I can pull any economic indicator and immediately have all the countries along\u00a0with all the years.<\/p>\n
Now I can easily visualize 4 selected indicators. For nicer graphs import Seaborn and set the color palette so that each line on the graph was plotted with a different color. Try comparing charts with and without Seaborn.<\/p>\n
Drawing directly\u00a0with\u00a0the pandas is really simple – just for our pivot table choose the interesting indicator, then transpose the data (function .T) and plot (, plot ‘).<\/p>\n
import seaborn as sns\r\npalette = sns.color_palette(\"Paired\", 10)\r\nsns.set_palette(palette)\r\n \r\npv2.loc['GDP (current US$)'].T.plot(alpha=0.75, rot=45)\r\npv2.loc['GDP per capita (current US$)'].T.plot(alpha-0.8, rot=45)\r\npv2.loc['GDP per capita (current US$)'].T.plot(alpha=0.75, rot=45)\r\npv2.loc['GDP growth (annual %)'].T.plot(alpha=0.75, rot=45)<\/pre>\n
The first two charts<\/p>\n
    \n
  1. GDP (current US$), data from World\u00a0bank<\/li>\n<\/ol>\n
    \"\"<\/a><\/p>\n
    2.\u00a0GDP per capita, data from World\u00a0bank<\/p>\n
    \"\"<\/a><\/p>\n
    Let’s try to perform a simple regression from the GDP data to see if there is a chance that one day we can catch up with Germany. This time we will use the “lmplot” function from the Seaborn library, except that the data must lead to a form of time series.<\/p>\n
    From the data in the form of a table with countries as columns, we need to create a table in which we will have only three columns [years, the country GDP]. We do this through a series of operations, the removal of the\u00a0index,\u00a0because our table at the beginning of the year is indexed (unique rows), changes of the name of the column. The key operation here is the “melt” function that transmits the data from the column and adds them into the next rows. So that we are able to make the following transformation. The attached images omitted part of the columns and rows but I hope its clear.<\/p>\n
    \"\"\u00a0\"\"<\/p>\n
    #seaborn plots\r\nplot_data = pv2.loc['GDP (current US$)'].T.reset_index()\r\nplot_data.rename(columns={'index':'Years'}, inplace=True)\r\n# unpivot the data, change from table view, where we have columns for each \r\n# country, to big long time series data, [year, country code, value]\r\nmelt_data = pd.melt(plot_data, id_vars=['Years'],var_name='Country')\r\nmelt_data.rename(columns={'value':'GDP'}, inplace=True)\r\nsns.lmplot(x=\"Years\", y=\"GDP\", hue=\"Country\", data=melt_data, palette=\"Set1\");<\/pre>\n
    We should get a result similar to this:<\/p>\n
    \"\"<\/a><\/p>\n
    Important links<\/span><\/p>\n