Introduction to GIS scripting

class: center, middle

# Introduction to GIS scripting
## Python introduction

INBO workshop, 9/11 May 2017

Stijn Van Hoey

https://github.com/stijnvanhoey/course_gis_scripting

---
class: center, middle

# Setup working environment

---

# Setting up...

* Working on JupyterHub:

* Log in at http://lifewatch.inbo.be/jupyterhub
        * Username: voornaam_achternaam
        * Paswoord: ...
  * Go to folder `course_gis_scripting/notebooks`

---
class: center, middle

When you see something like this...

![:scale 100%](./img/startup.png)

...relax, you're ready to start!

---
class: center, middle

![:scale 100%](https://i.ytimg.com/vi/PlaYMh-u-2w/maxresdefault.jpg)

---
class: center, middle

# Working with python

---
class: left, middle

## It is just plain text...

## ...until you run it!

---

## (I)python console

**Interactive mode**: It functions like an *advanced calculator*, <br>executing one command at a time:

<br>
.center[![:scale 75%](./img/ipython.png)]

---

## Python scripts

**Scripting mode**: Executing a series of commands saved in text file, with a `.py` extension after the name of your file.

.center[![:scale 75%](./img/script.png)]

---

## Interactive Development Environment (IDE)

Provides an environment to combine **Interactive mode** and <br> **Scripting mode**

.center[![:scale 55%](./img/spyder.png)]

--
<div class="alert alert-info">
<a href="https://pythonhosted.org/spyder/">Spyder</a> is the familiar environment for Matlab/Rstudio-users.
</div>

---

## Jupyter Notebook

**Jupyter notebook** provides an **interactive** scripting environment,<br> ideal for exploration, prototyping, tutorials,...<br>

.center[![:scale 70%](./img/notebook.png)]

--
<br>
<div class="alert alert-warning">
Comparable to Rmarkdown
</div>

---
class: middle, center

## Intermezzo!

---
class: middle, center

ask for help

![:scale 60%](./img/postit.jpg)

list your never-to-forget commands

---
class: middle, center

![:scale 100%](./img/issuetracker.png)

<div class="alert alert-danger">
Report bugs, typo's, suggestions... as issues: <a href="https://github.com/stijnvanhoey/course_gis_scripting/issues/new">New issue</a>
</div>

---

# Overview morning session

<br><br><br>
1. Getting to know the Jupyter notebook
1. Introduction to Python
1. Introduction to scientific Python: Numpy, Pandas

---
class: center, middle

# Jupyter notebook

`00-jupyter_introduction.ipynb`

---
class: center, middle

# Python: the essentials

`01-python-introduction.ipynb`

---
class: center, middle

# Python's scientific ecosystem

#### ## Thanks to Jake VanderPlas for the figure

---
class: center, middle, bgheader
background-image: url(./img/JakeVdP-ecosystem1.svg)
background-size: cover

---
count: false
class: center, middle, bgheader
background-image: url(./img/JakeVdP-ecosystem2.svg)
background-size: cover

---
count: false
class: center, middle, bgheader
background-image: url(./img/JakeVdP-ecosystem3.svg)
background-size: cover

---
count: false
class: center, middle, bgheader
background-image: url(./img/JakeVdP-ecosystem4.svg)
background-size: cover

---
count: false
class: center, middle, bgheader
background-image: url(./img/JakeVdP-ecosystem5.png)
background-size: cover

---

# And many more tools

**Performance**: Numba, Numexpr, Theano, Cython ...

**Visualisation**: Bokeh, Seaborn, Altair, Plotly, Mayavi, HoloViews, ...

**Data structures and (parallel/distributed) computation**: Xarray, Dask, Distributed, ...

**Packaging and distribution**: pip/wheels, conda, Anaconda, Canopy, ...

<br>
<div class="alert alert-info">
    <b>REMEMBER</b>:<br><br>

Feel free to ask me advice/support on the use of new packages!

</div>

---
count: false
class: center, middle, bgheader
background-image: url(./img/JakeVdP-ecosystem5bis.png)
background-size: cover

---
class: center, middle

# Scientific Python: 
# the essentials

`02-scientific-python-introduction.ipynb`

---
class: center, middle

# Why Python?

---
class: center, middle

# Python is...

---

## …OPEN SOURCE
* No license issues!

---

## …WIDESPREAD
* Large world-wide community ensures **fast development**
* Used in **scientific** and non-scientific institutes <br> (Google, NASA, ESRI, VMM,...)
* Python is a standard in **GIS** <br> (commercial and non-commercial tools)
* **Multiplatform** (linux, windows,…)

---

## …REPRODUCIBLE

.center[

instead of
```
what did you/I clicked a few months ago?’-style WYSISYG
```
]

---

## …REUSE

.center[

instead of
```
what did you/I clicked a few months ago?’-style WYSISYG
```
]

---

## …MULTI-FUNCTIONAL

* Research and development: the python *scientific ecosystem*
* Daily use: great as ductape/glue
* **Automation** of returning tasks

---

<div class="alert alert-warning">
<b>Waterinfo example</b><br>
from editing a script <code>waterinfo_example_1.py</code> towards a reusable functionality <code>waterinfo_example_2.py</code> inside Python and from the command line!
<center><img src="./img/waterinfo_example.png" width="600"></center>
</div>

Command line examples:

```python
python waterinfo_example_2.py 0162939042
python waterinfo_example_2.py 0163079042 --period P10D --windows 12
python waterinfo_example_2.py 023072043 --period P3D --windows 6 12 24
```

(check also [this tutorial](https://github.com/jorisvandenbossche/DS-python-data-analysis/blob/master/notebooks/workflow_example_evaluation.ipynb))

---

## SO WHAT? The Holy Grail ? No!
* Commercial packages give specific support  and more consistent help-functions (cfr. MATLAB, ArcGIS)
* Commercial packages can be more client-friendly
* For hard numercrunching stuff, NOT the fastest solution
* Fortran, C++ etc are also ‘widespread’, so python is just a language amongst the others
* Risk of getting lost in the forest/flood of ‘useful packages’

---
class: center, middle

# Conda

---

# Conda

### Why using (ana/mini)conda?

- Consistent package manager across Windows, Mac and Linux
- Many precompiled packages available
- ... *more technical arguments*...

<div class="alert alert-success">
<b>SUMMARY:</b> Easy to install packages!
</div>

### Why different environments?

- Manage the dependencies of a specific project/paper/group/...
- You can install different version of Python and other packages alongside on your computer
- Easily share environments with other people

<div class="alert alert-success">
<b>SUMMARY:</b> Make sure you work with the same packages!
</div>

---

# Conda: info

* [Anaconda Navigator info](https://docs.continuum.io/anaconda/navigator-using)
    * add channel: `conda-forge` (!)
* [Tutorials Anaconda Navigator](https://docs.continuum.io/anaconda/navigator-tutorial)

---

## Hungry?

* [Python for ecologists](http://www.datacarpentry.org/python-ecology-lesson/): introduction for people with no programming experience
* [Scipy lecture notes](http://www.scipy-lectures.org/) (!): Python, Numpy, Matplotlib, Scipy
* [Python for data analysis](https://github.com/jorisvandenbossche/DS-python-data-analysis): Pandas

---
class: center, middle

![:scale 100%](./img/pyutils.png)

---
class: center, middle

###Reading advice

[Good Enough Practices in Scientific Computing](https://arxiv.org/pdf/1609.00037v1.pdf)

> "*However, while most scientists are careful to validate their laboratory and field equipment, most do not know how reliable their software is*"

---
class: center, middle

## Good luck!

![](http://www.backwaterreptiles.com/images/pythons/angolan-python-for-sale.jpg)