Site icon Analytics4All

Python: Generators

Advertisements

I apologize in advanced as this topic is going to get a little computer sciency. Unfortunately there is no way around it. At first look, generators are going to resemble all the other iterables we have covered so far. But unlike loops we have used so far, generators produce iterables “lazily”.

So what do I mean by “lazily”?

Let’s consider this infinite loop I created below. This will run forever, producing a List (at least until your computer’s memory runs out). And that is the issue. Eventually the computer will run out of memory.

**note if you actually decide to run the code above, you will need to force it to stop. Just closing the notebook won’t do it. Go to File>Close and Halt to stop this loop from running and eventually crashing your PC.

This becomes are consideration when working with large data sets. The more memory you chew up, the worse you performance. So a work around is to use generators. Generators produce the data lazily, meaning they produce the iterator, yield it, and then forget it – they don’t put the values into a List like regular iterators do. They only yield one iterator at a time.

Notice my continued use of the world yield? There is a reason for that. Look at the code for a generator below:

Note that generators have to be functions. The yield command – which signifies a generator,  cannot be used outside of a function.

Try your code in our online Python console:  

Now, I know this looks like I am actually repeating work here. I mean I am using two loops to do what I could do with one loop. This issue arises when we have large numbers. If you are only iterating a few hundred numbers, you probably don’t need to worry about generators. However, if you are going to be iterating 100,000 elements, you will see some major performance improvements by using the code above.


If you enjoyed this lesson, click LIKE below, or even better, leave me a COMMENT. 

Last lesson: list comprehension

Next Lesson: regular expressions

Back to Python Course: Course

Exit mobile version