How to iterate through a python dictionary
A python Dictionary is one of the important data structure which is extensively used in data science and elsewhere when you want to store the data as a key-value pair. In this post we will take a deep dive into dictionaries and ways to iterate over dictionary and find out how to sort a dictionary values and other operations using dictionary data structure
Basically a dictionary in python is a mapping object where it maps a key to a value. The keys are hashable values which are mapped to values. The keys are arbitrary values and any values that are not hashable that is values containing list or other mutable types may not be used as Keys. Even it is not a good practice to use numeric values as keys also.
How to create a Dictionary?
We can create a Dictionary using key:value pairs separated by commas or using the dict constructor
Using comma separated key value pair
d = {
"name": "Charlie",
"age": 42,
"city": "NYC",
"state": "NY",
"rank": 1.0 }
Using dict constructor
d = dict( name= "Charlie", age= 42, city = "NYC", state = "NY",rank = 1.0)
Convert two list into a dictionary
d = dict( zip(['name','age','city','state','rank'],['Charlie',42,'NYC','NY',1.0]))
Convert list of tuples(Key,Value) into a dictionary
d = dict([("name","charlie"),("age",42),("city","NYC","state","NY"),("rank",1.0)])
What’s Changed for Dictionary in Python 3.6
Dictionaries got ordered in Python 3.6 that means it remembers the order of insertion of key and value pairs. It means that keyword arguments can now be iterated by their creation order, which is basically the cpython implementation of python
The memory usage of this new dictionary implementation will also reduce the memory usage by 20-25%
Here is an example from the python dev mailing list for the implementation of new dict in python 3.6
Create a function to get the dictionary keys
def func(**kw):
print(kw.keys())
Calling the above function in Python 3.5 and before returns an un-ordered dict_keys. Check the output the keys are randomly ordered
func(a=1, b=2, c=3, d=4, e=5)
Output in Python 3.5 and before:
dict_keys(['c', 'd', 'e', 'b', 'a']) # Keys are randomly ordered
Calling the same function in python3.6 and above returns a dict_keys in the same order as it has been passed in the function
Output in Python 3.6 and above:
dict_keys(['a', 'b', 'c', 'd', 'e']) # Keys are ordered
Iterating thru a dictionary
As a python developer or data scientists you will be working on dictionaries a lot and there are calculations or actions that needs to be performed while working through the dictionary keys and values
In this section we will see what are the ways you can retrieve the data from the dictionary
Python supports a concept of iteration over containers and An iterator needs to define two methods: iter() and next(). Usually, the object itself defines the next() method, so it just returns itself as the iterator.
the iter defines the next method which will be called by statements like for and in to yield the next item, and next() should raise the StopIteration exception when there are no more items
Hope this clears how the iterator works on the dictionary
These methods are used by for and in statements, so what it means is if you put a dictionary under a for loop then it will automatically call the iter() method and iterate over the dictionaries keys
Iterate dictionary using keys
for keys in d:
print(keys, ":", d[keys])
Output:
name : charlie
age : 42
city : NYC
state : NY
rank : 1.0
Dictionary view objects
This provide a window of the dictionary entries and when any of the item changes in the dict then it will reflect those changes in the views
As per the python official documentation:
The objects returned by dict.keys(), dict.values() and dict.items() are view objects. They provide a dynamic view on the dictionary’s entries, which means that when the dictionary changes, the view reflects these changes
Length of Dictionary
This returns number of key,value pairs in the dictionary
len(d)
list of dictionary keys
the keys() function returns dict_keys() which is wrapped with the list to get a list object.
list(d.keys())
Output:
['name', 'age', 'city', 'state', 'rank']
list of dictionary values
the values() function returns dict_values() which is wrapped with the list to get a list object
list(d.values())
Output:
['charlie', 42, 'NYC', 'NY', 1.0]
Iterate thru dict values
The dict_values is an iterable so you can directly iterate through the values of dictionaries using for loop
for value in d.values():
print(value)
Output:
Charlie
42
NYC
NY
1.0
Iterate thru dict keys
The dict_keys is an iterable so you can directly iterate through the keys of dictionaries using for loop
for key in d.keys():
print(key)
Output:
name
age
city
state
rank
Iterate key, value pair using dict.items
This returns a (key,value) pair which is a tuple object.
for key,value in d.items():
print(key,":",value)
Output:
name : Charlie
age : 42
city : NYC
state : NY
rank : 1.0
Dictionary Comprehension
It is a syntactical extension to list comprehension. It produce a dictionary object and you can’t add keys to an existing dictionary using this
You group the expression using curly braces and the left part before the for keyword expresses both a key and a value, separated by a colon
city = ['NYC', 'Boston', 'Los Angeles']
state = ['NY', 'Massachussets', 'California']
city_state_dict = {key: value for key, value in zip(state, city)}
Output:
{‘California’: ‘Los Angeles’, ‘Massachussets’: ‘Boston’, ‘NY’: ‘NYC’}
Dictionary Get Key
get(key,default)
This method returns the value for key if key is in the dictionary, else default. If default is not given, it defaults to None, so that this method never raises a KeyError.
Let’s understand this with a small example
Here is a dictionary of city and respective states:
> city_state_dict = {
> "NYC": "New York",
> "Boston": "Massachussets",
> "Charlotte": "North Carolina",
> "Los Angeles": "California",
> "Minnetonka": "Minnesota" }
We have to output the state name when user enters a city
When city is in Dictionary Key:
city = "Charlotte"
print(f'You live in state of {city_state_dict.get(city)}')
Output:
You live in state of North Carolina
When city is not in Dictionary Key:
city = "Raleigh"
print(f'you live in state of {city_state_dict.get(city)}')
Output:
You live in state of None
Unpack a Dictionary using itemgetter
You can unpack a dictionary using operator.itemgetter() function which is helpful if you want to get the values from the dictionary given a series of keys
Look at this example below we are extracting the values of keys a, b and c using itemgetter
from operator import itemgetter
d = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
a, b, c = itemgetter('a', 'b', 'c')(stuff)
print(a,b,c)
Sorting
There are certain situations where you need to sort the dictionary either by key or values. You can achieve this using sorted() function. In the below section we will see how to sort the dictionary by keys and values
if you are using python 3.6 and above then do not worry the dictionary are ordered data structured and can be easily sorted using sorted() function inside a dictionary comprehension
Sorting Dictionary by Keys
You can pass the entire dictionary dict_items as an argument to sorted function and it will return a list of tuples after sorting which can be converted back to dictionary using the dict constructor
d = {'B':10,'A': 4}
dict(sorted(d.items()))
Output:
{‘A’: 4, ‘B’: 10}
You can see the output the dictionary keys are sorted alphabetically
Sorting Dictionary by Values
You can also sort the dictionary with their values using the sorted() function and another argument key and value of the key parameter should be a function that takes a single argument and returns a key to use for sorting purposes
By default it gives the result in ascending order. You can see the list of keys that is returned here based on the values of each of the corresponding keys arranged in ascending order
Ascending Order
d = {'a':1000, 'b':3000, 'c': 100}
sorted(d, key=d.get)
Output:
['c', 'a', 'b']
if you want the list of keys to be in descending order above then add another argument called reverse as True and this will give the keys based on their values arranged in descending order
Descending order
d = {'a':1000, 'b':3000, 'c': 100}
sorted(d, key=d.get, reverse=True)
Output:
['b', 'a', 'c']
Enumerate Dictionary
You can also enumerate through the dictionary and can get the index of each of the key
In the final output you can see the index for each of the key .
Just remember d.keys(), d.values() returns a dict_keys object which behaves a lot more like a set than a list
Therefore, dict.values() needs to be wrapped in a list. You can see in the below code we have used list(d.values())[index]
d = {'a':1000, 'b':3000, 'c': 100, 'd':542,'e':790, 'f': 1042}
for index,item in enumerate(d):
print(index," ",item," : ",list(d.values())[index])
Output:
0 a : 1000
1 b : 3000
2 c : 100
3 d : 542
4 e : 790
5 f : 1042
Filter Dictionary
You can filter a dictionary using the dictionary comprehension which will create a new dictionary based on the filter criteria.
In this example we are filtering out all the key-value pairs where the value is < 500. The final dictionary have all the keys with values greater than 500
d = {'a':1000, 'b':3000, 'c': 100, 'd':542,'e':790, 'f': 1042}
{k:v for k,v in d.items() if v > 500}
Output:
{‘a’: 1000, ‘b’: 3000, ‘d’: 542, ‘e’: 790, ‘f’: 1042}
Update Dictionary value
The update method overwrites the existing keys and Update the dictionary with the key/value pairs
It accepts either a dictionary object or an iterable of key/value pairs
d = {'a':1000, 'b':3000, 'c': 100, 'd':542,'e':790, 'f': 1042}
d.update(g=790,b=3049)
Output:
{‘a’: 1000, ‘b’: 3049, ‘c’: 100, ‘d’: 542, ‘e’: 790, ‘f’: 1042, ‘g’: 790}
Delete Dictionary Key
you can use del to delete a key and it’s value from a dictionary if you are sure key exists in the dictionary else it Raises a KeyError if key is not in the map
d = {'a':1000, 'b':3000, 'c': 100, 'd':542,'e':790, 'f': 1042}
del d['f']
Output:
{‘a’: 1000, ‘b’: 3000, ‘c’: 100, ‘d’: 542, ‘e’: 790}
dict.pop()
If you aren’t sure about the key exists in dictionary then use dict.pop().
This will return d[‘f’] if key exists in the dictionary, and None otherwise.
If the second parameter is not specified (i.e. d.pop(‘f’)) and key does not exist, a KeyError is raised.
d.pop('f', None)
Merge two or more Dictionaries
You can use the dictionary unpacking operator ** to merge two or more dictionary into one single dictionary and then you can iterate this new merged dictionary
Here is an example:
city_1 = {'NYC':'NY','Boston':'Massachussets'}
city_2 = {'Los Angeles': 'California', 'Charlotte': 'North Carolina'}
# Merge Two Dictionaries
merge_city_1_2 = {**city_1,**city_2}
Output:
{‘NYC’: ‘NY’, ‘Boston’: ‘Massachussets’, ‘Los Angeles’: ‘California’, ‘Charlotte’: ‘North Carolina’}
Conclusion:
We have reached to the end of this comprehensive and detailed post on iterating through dictionaries. So here is what we have learned so far through our journey thru this blog
- What are Dictionaries and how it works and what are different ways to create a dictionary in python
- Dictionary view objects, why it is important and it’s dynamic nature to reflect the changes immediately
- Dictionary get function to retrieve the values from a simple and nested dictionary and it returns None if the key is not found
- How to sort a dictionary by it’s keys and values in ascending and descending orders
- Enumerate thru a dictionary and get the index position of it’s keys which are ordered now after python 3.6
- Finally, what are the different methods to update and delete a dictionary key
I have tried to cover all the possible ways to iterate through a dictionary and explain different ways to work with dictionary efficiently.
Still if there are anything you feel should be included in this post or can be done in more optimized way then please leave a comment below.