Tag: python for data science

Python: Error handling

On By Ben Larson Ph.D.In Python, Python: Learn Python Course1 Comment

No matter how well crafted your code, you will inevitably come across an error every once in a while. Unfortunately, Python’s default setting when catching an error is to crash the program.

In the example below, I accidentally asked for a letter, when the code int(input()) is looking for an integer.

So when I enter a string, I get an error

Try your code in our online Python console:  

Try

Using Try, we can handle such mishaps in a more elegant fashion

The syntax is a follows:

try:
        Your Code
except: 
         What to do if your code errors out
else:
         What to do if your code is successful

And as you can see from above, it doesn’t just protect against code errors, but it protects against user errors as well.

Using a while loop, we can give the user another chance if they don’t enter a number the first time.

Code explanation

  • while True:  – starts an infinite loop
  • continue – returns to the beginning of the while loop
  • break – exits the loop

Try your code in our online Python console:  

Finally

You can add the finally: command to the end of a try: statement if you have something you want to execute regardless of whether there was an error or not.

Try your code in our online Python console:  

Exception types:

You can also determine what your program does based on the type of error

ValueError

ZeroDivisionError

No Error

Try your code in our online Python console:  

A list of Exception types can be found here: Link

If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Last Lesson: Enumerate() and Sort()

Next Lesson: Lambda, map, reduce, filter

Back to Python Course: Course

Python: Numpy

On By Ben Larson Ph.D.In Python1 Comment

First off, CONGRATS for making it this far. Numpy really signifies the first step in real data science with Python.

Numpy is a library that adds advanced mathematical capabilities to Python. If you are using the Anaconda release of iPython, you already have numpy installed. If not, you will need to go over to their site and download it. Link to numpy site: www.numpy.org

Numpy adds multi-dimensional array capabilities to Python. It also provides mathematical tools for dealing with these arrays.

Array

What is an array? In different programming languages, arrays are equivalent to lists in Python. In fact, a single dimension array in Python can be used interchangeably with lists in most cases.

What makes arrays special in Python is the ability to make multidimensional arrays. Those who have taken math courses like Linear Algebra will better know multidimensional arrays by the term matrix. And if you paid attention in Linear Algebra, you will know there is a lot of cool things you can do with matrices.

For those who didn’t take Linear Algebra (you probably went out and made friends and had a social life), don’t worry, I will fill you in on what you need to know.

Numpy

Let’s start by creating a single dimension array

  • import numpy as np   — imports numpy into your notebook, we set the alias to np
  • x = np.array([8,7,4])  — array is a method of np, so the syntax is np.array. **note the [] inside the ()
  • x[1] = you call array just like lists
  • x.shape = shows you shape of your array, since we are single dimension, the answer is a single number – **note 3L – the L indicates integer in Python

pythonnumpy.jpg

Now let’s make a 2 x 3 matrix. When describing a matrix (or multi-dim array) the first number indicates the number of elements across and the second number indicates down. This is standard mathematical notation.

You can see I create a new row by enclosing the number sets in [] separated by a ‘,’

Now when want to call an item from the multi-dim array, you need to use 2 index numbers. y[0,1] = y[row, column]

pythonnumpy2

np.arange()

np.arange() is a command we can use to auto populate an array.

  1. np.arang(5) –create a one – dim array with 5 elements
  2. np.arange(10).reshape(2,5) – reshape lets you convert a one-dim array into a multi-dim array

pythonnumpy4

Transpose

Transposing a matrix means flipping it on its axis. This comes in handy in applications like multi-variate regressions. To transpose our array in Python, we use the “.T” method

pythonnumpy5.jpg

dot Product

Performing a dot product on a matrix is a useful calculation, but it is very time consuming. Numpy makes it easy though.

pythonnumpy6.jpg

If you need a brush up on dot products, this is a great link: matrix multiplication

If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Last Lesson: Create. Import, and use Modules

Next Lesson: Numpy Part II

Return to: Python for Data Science Course

Follow this link for more Python content: Python

 

 

Python: **Kwargs and *Args

On By Ben Larson Ph.D.In Python, Python: Learn Python Course1 Comment

Outside of beings incredibly fun to say, Kwargs and Args can be pretty useful.

Here is how they work. Let us say you wanted to make a function that adds 4 numbers:

Notice what happens when we only provide 3 arguments, we error out. The same thing happens if we try to give the function 5 arguments.

pythonkwa

Well, if the world perfectly predictable, this wouldn’t be an issue. But in the real world, making a function that adds 4 and only 4 numbers isn’t very useful. A function that can add any given set of numbers however is very useful.

*args

*args to the rescue

Using the keyword *args inside the function parenthesis gives you the flexibility of having as many, or as few arguments as you want.

pythonkwa1.jpg

Try your code in our online Python console:  

**kwargs

Kwargs are basically args with keywords. Think dictionaries.

pythonkwa2

You can combine regular arguments with *args and **kwargs

Try your code in our online Python console:  

If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Last lesson: Regular expressions

Back to Python Course: Course

 

Python zip and unpack

On By Ben Larson Ph.D.In Python, Python: Learn Python Course1 Comment

zip

Zip is a quick way to take multiple lists can combine them.

unpacking

To reverse the effect, turn a list of tuples into separate lists, use * in the parenthesis: zip(*x)

Try your code in our online Python console:  

If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Last Lesson: lambda, map, reduce, filter

Next Lesson: list comprehension

Back to Python Course: Course

 

Python: Regular Expressions

On By Ben Larson Ph.D.In Python, Python: Learn Python CourseLeave a comment

Regular Expressions are used to parse text. In the world of Big Data, being able to work with unstructured text is a big advantage.

To use regular expressions, first you must import the module. This is done by placing the command import re at the top of your code.

re.search()

Now, let us examine this code below:

We want to see if dog (assigned to x) is in the sentence ‘I just a saw dog. He was chasing a cat.'(assigned to y)

Using the search() method from re, we ask if re.search(x,y). Note you place the item you are searching by first in the parenthesis. re.search() returns a boolean value (True, False).

You can use re.search with lists of search items as well.

Here z is taking one item from the list x at a time and running it through re.search. Notice ‘one’ returns True, while ‘two’ returns false.

Try your code in our online Python console:  

re.findall()

re.findall returns all instances of your search term. Notice it found water whether it was a stand alone word, or part of a larger word.

pythonreg2.jpg

re.split()

The re.split() method does pretty much what you would think it does. You can pick a delimiter and the method will split your string at that delimiter.

In the example below, ‘;‘ is my delimiter. Notice how it split my string in two, plus removed the delimiter for me.

pythonreg3

Try your code in our online Python console:  

use re.search() to find position

You can use re.search() to find the starting and ending position of a search item in a string

pythonreg4

exclusion

If you want to exclude characters, use the ^ between square brackets [].

This example excludes the letter s = [^s] and puts the remaining characters in a list

In the second example, I add + after the []. This keeps all the characters together.

pythonreg5.jpg

This next example is a useful tool you will find yourself using in text mining. Here we use [^?!. ]+ to remove punctuation.

pythonreg6

Try your code in our online Python console:  

If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Last Lesson: Generators

Next Lesson: kwargs and args

Back to Python Course: Course

 

Python: Generators

On By Ben Larson Ph.D.In Python, Python: Learn Python Course1 Comment

I apologize in advanced as this topic is going to get a little computer sciency. Unfortunately there is no way around it. At first look, generators are going to resemble all the other iterables we have covered so far. But unlike loops we have used so far, generators produce iterables “lazily”.

So what do I mean by “lazily”?

Let’s consider this infinite loop I created below. This will run forever, producing a List (at least until your computer’s memory runs out). And that is the issue. Eventually the computer will run out of memory.

pythonGen

**note if you actually decide to run the code above, you will need to force it to stop. Just closing the notebook won’t do it. Go to File>Close and Halt to stop this loop from running and eventually crashing your PC.

pythonGen1

This becomes are consideration when working with large data sets. The more memory you chew up, the worse you performance. So a work around is to use generators. Generators produce the data lazily, meaning they produce the iterator, yield it, and then forget it – they don’t put the values into a List like regular iterators do. They only yield one iterator at a time.

Notice my continued use of the world yield? There is a reason for that. Look at the code for a generator below:

Note that generators have to be functions. The yield command – which signifies a generator,  cannot be used outside of a function.

pythonGen2.jpg

Try your code in our online Python console:  

Now, I know this looks like I am actually repeating work here. I mean I am using two loops to do what I could do with one loop. This issue arises when we have large numbers. If you are only iterating a few hundred numbers, you probably don’t need to worry about generators. However, if you are going to be iterating 100,000 elements, you will see some major performance improvements by using the code above.

If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Last lesson: list comprehension

Next Lesson: regular expressions

Back to Python Course: Course

Python: List Comprehension

On By Ben Larson Ph.D.In Python, Python: Learn Python Course4 Comments

List comprehensions are a method for taking a list of elements and performing a transformation on each element.

In our first example, we want to take numbers 0-9, square them, and have the result end up in a list.

ln[5] shows how you would perform this task using for a loop to iterate.

ln[1] does the same thing, but it does it in one line.

 

pythonListcomp

Try your code in our online Python console:  

As you can see, the list comprehension basically crunches the for loop into one line. The syntax is simple enough:

S            =  [x**2 for x in range(10)]

Assign a variable = [operation for loop]

Find even numbers in a list:

Here we add an if statement inside the iteration. (x%2 is modulus, meaning it returns remainder. So 4%2 returns a remainder of 0 and 5%2 returns are remainder of 1)

pythonListcomp1.jpg

You can use list comprehensions with more complex formulas:

pythonListcomp3

You can even use functions from within a list comprehension

pythonListcomp4.jpg

Try your code in our online Python console:  

If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Last Lesson: Zip and unpack

Next Lesson: Generators

Back to Python Course: Course

 

Python: Install Python

On By Ben Larson Ph.D.In PythonLeave a comment

 

While Python 3 has been out now for a while, Python is not backwards compatible and most data science based libraries run on old reliable Python 2.7.  So that is the version I will be teaching.

If you are an advanced computer user, you can run both versions of Python on your computer simultaneously. Feel free to do so if you want to see the differences in the versions. While version 3 is  becoming more widely used in some areas, most of the libraries and information you will find involving Data Science or Analytics will still use version 2.7.

Downloading Python

Now, you can always download Python at http://www.python.org. However, I recommend downloading Anaconda. This distribution comes with more than 400 of the more popular Python packages in math, science, engineering, and more importantly – data analysis. The link to download Anaconda is: Anaconda Download.

For detailed installation instructions: Anaconda Install

Again, you do not need to use the Anaconda distribution of Python, but it will make following along with my tutorials much easier.

Another great advantage of Anaconda is that is comes with iPython already installed, which is a very popular IDE used by Data Scientists.

Running Python

Once you have Anaconda installed, open the Anaconda Prompt

pythonInstall.jpg

It will open like a Command Prompt / Terminal Window

At the prompt type: jupyter notebook

The Jupyter Notebook will open your default browser.

To start using Python, go to New in the upper right corner and select Python 2

pythonInstall1.jpg

Double Click Untitled at the top of your new notebook to change the name. Let’s  call this one Fundamentals

pythonInstall2

pythonInstall3

Using the Notebook

In Jupyter Notebooks, we work in the shaded rectangles marked In[]. To see the output of your command, you press Shift+Enter. Enter alone adds another line to the code block you are working on, but does not execute the code.

pythonInstall4

** note in the second example, Python only executes 1-2. Make sure when you have separate executable, you hit Shift+Enter each one.

 

If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Follow this link for more Python content: Python

Next: Fundamentals