Here is a code block to create a database if you want to play along
create database sandbox;
use sandbox;
CREATE TABLE employee_id (
emp_nm varchar(30) not null,
emp_id varchar(8),
b_emp_id varchar(8),
PRIMARY KEY(emp_nm) );
While loading data from Excel files and CVS files is pretty straightforward in Python, getting database from a SQL database is really a skill you should learn. SQL databases can store much more data than an Excel file and are used pretty much everywhere now.
This lesson will focus on MySQL, I have another lesson for SQL Server as well. The syntax does vary a little and if you are working with Oracle or Teradata, your syntax may change a bit as well. Luckily the Internet has plenty of resources to point you in the right direction.
Now, for MySQL, lets get started. First you are going to want to install mysql.connector (if you have the Anaconda distribution that I recommend, it comes with this pre-installed.
If you don’t have it installed, you can get it using pip or conda – I have a video showing you how to install a module here: video link — skip ahead to the 7 minute mark where I explain how to install modules
Once install, import it into your notebook
Now lets make a connection to our server: syntax should be self explanatory – localhost simply means its installed on my computer — otherwise you would provide a network path to the server there.
Let’s start by looking around- First I want a name off all the databases in my instance
Now in this example we will be working with the Sandbox database — I provided code at the top of the lesson you can paste and run in your MySQL instance to create a Sandbox database
Now lets add a new element to our connection string – database
And query the table names in the Sandbox database
Using the .execute() method, we can pass SQL commands to the MySQL instance
Below I am creating a table -> passing data to the table -> and committing the data
Without the commit() command, the data will not be saved into the table.
To add multiple rows to the table, I can use the executemany() method
Now let’s query our table. Note the .fetchall() method — this brings in all the rows from the query. I can iterate through list by running a simple for loop
I can also use the command .column_names after my cursor to return the column names of the last table I queried
Finally, we can use Pandas to put this database table into a dataframe we can work with
This is a more advanced programming topic, and honestly not one I make much use of myself. However, I got a request to make a lesson on Closures and Decorators. I am going to make it into two lessons to try to make it a bit clearer.
Now to understand closures and decorators, the first thing we need to discuss is nested functions. In programming, nested functions means wrapping a function inside of another function. Here is a very simple example of a nested function:
Now if I just call this with outerFunction(‘Hello World’), the argument x gets passed into the innerFunction which is in turn called by the outerFunction. The result is shown below:
Now why would anyone do this? Outside of just being able to show you that you can? I don’t know. There are reasons for nested functions that involve hiding variables from the code outside of the functions. That is a bit beyond the scope of this lesson. I will try to tackle it when I create my Object Oriented Programming lessons.
For now, let us get on to Closures. Let’s look at the code below:
It is very close to the first example, except notice 2 little changes. First, instead of just calling the innerFunction at the end of the function, we are returning it. Notice there is no () when we return innerFunction
Also notice we are passing the outerFunction to a variable and when we call the var we add () to the end: var()
What we have done now is created an official Closure, let me show you why that matters below.
First, notice I delete the outerFunction. When I try calling it returns an error saying outerFunction is not defined (meaning it doesn’t exist).
But, look what happens when I called var(). It is still there. The closure commits the value to memory when you create it. That is not something that would happen from a normal function or even nesting function.
SO WHY THE HELL SHOULD I CARE????
So why this is cool, and why you might use this in the future, is imagine a function that does processer heavy multi-step calculations. If you just keep calling the function as usual, each time you call it, it has to run the whole chain of calculations again. However, once you’ve called it as closure, you don’t have to run the calculations again, you just get the value.
So, how about a more concrete example
Here I created a closure that uses two functions which each take 1 argument, the arguments are then multiplied together.
Notice, I created a variable called double where I sent the outer function mult_by() a value of 2.
When I call double() I can pass it a value for (y)
I can create as many of these instances as I want. Look below, created one for Triple, and notice double is still functional
I’ll continue this in the Decorators lesson.
Just the main thing to keep in mind about closures, is that they improve program performance by no requiring the function to be run each time.
If you are still a little confused after reading this, don’t feel bad. It took me a few tries to finally understand closures myself.
Welcome to Python for Data Science, my free course that will take you from complete beginner to being able to build a machine learning model. The course will consist of 4 modules, the first one: Fundamentals is a available now. Each lesson contains a write up as well as a video. There is over an hour of videos for the Fundamentals Module.
Each module will also also contain some projects: some are simple challenges while the later projects will require you to build a ML model and test it.
The code for each lesson and project solutions are available to download below each module.
If you are working from your phone/tablet or work computer where you can’t install Python, I have a browser based Python console you can work with here:
For your first project in the course, I am giving you the code below. You will notice I am placing 2 dictionaries in a list:
d = {'Name': 'Ben', 'Age': 35}
e = {'Name': 'Christine', 'Age': 30}
a = []
a.append(d)
a.append(e)
print(a)
Your challenge, should you choose to accept it, will be to run this code on your machine or in the test python browser (click on the blue arrow below)
Try your Python code in the free console
I then want you to
1) Print out the second dictionary from the list
2) Print out the name from the first dictionary
3) Print out both ages
Note: I have not shown how to work with dictionaries inside a list. So consider this a stretch project. Don’t be afraid to use Google to help find an answer.
Project Answer Notebook Download Available at Course Page below:
I want to start with the obvious. there are many different version of Python, and many platforms you can use to work with it, from command line/terminal, IDEs like Spyder of IDLE, or notebooks like Jupyter. For this course on Python, I have chosen to start with Jupyter Notebooks, because I feel this is a great tool for learning Python.
Installing Python
First things first. You need to get Python on your machine. In this series of lessons, I am going to use the Anaconda Distribution. I am choosing this distribution for one simple reason: it is built for data science, most of the tools you will need for data analysis and machine learning come pre-loaded with Anaconda.
To install this distribution, go to the Anaconda website and install Python: Anaconda
Jupyter Notebook
Once installed, you will find Jupyter Notebook in your program list.
Jupyter will open in your default browser and should look something like this
Click on the New button and then Python 3
You will now have a new notebook that looks like this
Lets start with some very basic code. We will do the traditional Hello World exercise
The code is a follows
print("Hello World")
Print() with the text you wish to print in quotes inside the parenthesis
After you type it into a notebook command box, click the arrow icon to run your code (or hit Shift+Enter)
