[Python Learning Log] The Magic of NumPy: Trying out Universal Functions (ufunc)! Goodbye, for-loops
[Introduction]
Hello! I am currently working hard on learning data analysis with Python. This time, I studied one of the most powerful features, **universal functions (ufunc)**, from Chapter 4, "NumPy Basics," of my favorite book, "Python for Data Analysis." The learning scope covers pages 107 to 110.
Until now, I thought it was standard to use for-loops whenever I wanted to perform the same calculation on each element of data, but it seems there is a much smarter and incomparably faster way in the world of NumPy. Through a dialogue with AI, I started with simple code practice and ended up exploring the deep realm of computer science regarding how random numbers work. I am leaving this record as a log of that learning journey.
[Learning Goal]
The learning goal for this session is simple yet powerful.
"Mastering **universal functions (ufunc)**, the magic tools in NumPy that allow you to finish a large number of calculations in an instant!"
Once you master this, you will be able to execute calculations such as square roots (x
), exponents (ex), and maximum value comparisons at ultra-high speeds without having to write for-loops for every single piece of data contained in an array (ndarray).
[Process and Discoveries]
Unary ufuncs: Starting with the basics
The first step is the "unary ufunc," which takes a single array as an argument.
Following the textbook, I first created an array containing integers from 0 to 9. This was easily done with np.arange(10).
Python
import numpy as np
arr = np.arange(10)
For this array 'arr', I tried using np.sqrt() to find the square root of all elements and np.exp() to calculate the exponent of all elements.
Python
# 各要素の平方根を計算
print(np.sqrt(arr))
# 各要素をeの指数として計算
print(np.exp(arr))
The execution result returned a new array with all calculations completed in an instant. If I had tried to do this using only standard Python features, it would have been a hassle to use a for-loop to extract each element one by one, calculate it, and add it to a new list. I was impressed by the simplicity and speed of ufuncs from the very beginning.
Binary ufuncs and learning from errors
Next, I challenged myself with a 'binary ufunc,' which takes two arrays as arguments. Here, I used np.maximum(), which compares elements at the same position in two arrays and creates a new array containing only the larger values.
First, I stumbled a bit while creating the two random arrays, x and y, to be compared.
Python
# 最初のコード（間違い）
x = np.random.random(8)
y = np.random.random(8)
print(np.maximum(x,y))
When I had an AI review it, it pointed out, 'Close! The textbook uses np.random.randn(), which follows a standard normal distribution.' This sparked a question.
Me: 'Are random and randn different?'
This question helped clarify the difference between the two functions.
Function Name | Range of Generated Values | Distribution (How data varies) | np.random.random | 0.0 or more, less than 1.0 | Uniform distribution (every value has the same probability) | np.random.randn | Minus to plus infinity | Standard normal distribution (many values near 0, fewer towards the ends)
In data analysis, randn is often used because we frequently deal with 'normal distributions' where data clusters around an average value, such as measurement errors or natural phenomena. I understood this, and corrected the code.
Python
# 正しいコード
x = np.random.randn(8)
y = np.random.randn(8)
print(np.maximum(x, y))
I successfully obtained an array that extracted the maximum value for each element from the two arrays. It was a good experience where errors and questions led to deeper knowledge.
Ufuncs that return multiple arrays
Finally, I tried np.modf() as a slightly unusual ufunc. This function takes a single array as an argument and decomposes each element into its 'fractional part' and 'integer part,' returning them as two separate arrays.
Python
arr2 = np.random.randn(7) * 5
print(arr2)
# 出力例: [ 3.84661053 1.51984874 5.09042654 ... ]
rem, whole = np.modf(arr2)
print(rem) # 小数部分の配列
# 出力例: [ 0.84661053 0.51984874 0.09042654 ... ]
print(whole) # 整数部分の配列
# 出力例: [ 3. 1. 5. ... ]
It is interesting that a single function can return two results in a tuple format. Through this exercise, I shared the following thoughts with the AI.
Me: "It is certainly interesting. If I use random numbers to perform multiplication or addition, it seems like I could do various calculations based on that data, which is fascinating."
I felt once again that this sense of "interest" is truly important for continuing to learn.
[Final Deliverable]
This is the code I created during this series of learning sessions to verify the operation of each ufunc.
Python
import numpy as np
# --- 単項ufunc ---
arr = np.arange(10)
# 平方根
print("--- np.sqrt ---")
print(np.sqrt(arr))
# 指数
print("\n--- np.exp ---")
print(np.exp(arr))
# --- 二項ufunc ---
x = np.random.randn(8)
y = np.random.randn(8)
# 最大値
print("\n--- np.maximum ---")
print(f"x: {x}")
print(f"y: {y}")
print(f"result: {np.maximum(x, y)}")
# --- 複数の配列を返すufunc ---
arr2 = np.random.randn(7) * 5
# 整数部分と小数部分
print("\n--- np.modf ---")
print(f"original: {arr2}")
rem, whole = np.modf(arr2)
print(f"remainder: {rem}")
print(f"whole_part: {whole}")
[Learning and Future Prospects]
What I learned (Technical points)
The power of ufuncs: You can perform mathematical calculations (square roots, exponents, comparisons, etc.) on all elements of an array at once without using for-loops. The code becomes concise, and the processing speed is overwhelmingly faster.
Types of ufuncs: I learned that there are various types, such as those that take one array, unary ufuncs (np.sqrt), those that take two, binary ufuncs (np.maximum), and ufuncs that return multiple arrays (np.modf).
Difference between np.random.random and np.random.randn: The former generates random numbers following a uniform distribution from 0 to 1, while the latter follows a standard normal distribution with a mean of 0 and a standard deviation of 1. I understood that randn appears frequently in the context of data analysis.
Application examples and reflections (Discoveries from the dialogue log)
This learning session did not end with just coding practice. In my dialogue with the AI, my simple questions became gateways to unexpected exploration.
Application to number theory: Seeing np.modf separate integers and decimals, I commented, "This seems useful for number theory in mathematics." The AI then taught me that other ufuncs like np.mod (remainder) and np.floor (rounding down) are also powerful tools in number theory for calculations like congruences and Gaussian symbols. The perspective that NumPy can be utilized in the realm of pure mathematics was a major discovery.
The nature of random numbers: My question, "How are random numbers created?" developed into a discussion about how computers generate pseudorandom numbers. The mechanism is that if you have an initial value (seed) and a calculation formula (algorithm, e.g., Mersenne Twister method), you can reproduce the same sequence of random numbers. Furthermore, when I asked, "So, can you reverse-calculate them?" the answer was, "Theoretically possible, but practically almost impossible because the internal state is huge and complex." Ultimately, I was able to learn about the existence of true random numbers that utilize unpredictable physical phenomena (such as thermal noise), allowing me to touch upon the depth of computers.
Reflections on Learning Methods (Discoveries from Dialogue Logs)
This time, by learning while interacting with an AI, I experienced the excitement of knowledge expanding like a chain reaction from a single theme.
Valuing the "Why?": Wondering about the difference between random and randn, and becoming interested in how random numbers work. By throwing these "why?" curiosities at the AI, I was able to gain living knowledge that goes beyond what is written in textbooks.
Errors are Opportunities for Learning: Even a small error, like correcting random to randn, became a perfect opportunity to learn how to distinguish between the two functions.
Dialogue Deepens Thinking: From a casual remark I made about it being "useful for number theory," the AI provided concrete application examples. Through this exchange, I felt my own thoughts being organized and deepened further.
It was a very dense learning session where I was able to learn everything from a single feature called ufunc to the basics of computer science, such as how random numbers work. I want to continue exploring while enjoying my questions in this manner.