Operators in Python are turned into “magic” method calls: x + y # becomes x.__add__(y) So: Different errors from different methods! The post Python operator overloading: Why x + y and y + x give different errors appeared first on LernerPython.

Because Python functions’ defaults are kept in __defaults__, avoid mutable defaults: add1.__defaults__ # ([1, 1, 1],) The post Mutable default arguments in Python: why `__defaults__` bites appeared first on LernerPython.

When a Python generator yields, its stack frame remains — inspect it for the current line and local variables: The post Python generator frames: Inspecting locals with gi_frame appeared first on LernerPython.

AI is the greatest technological revolution of our lifetimes, with far-reaching implications — both very good and very troubling — for every part of human society. As someone who teaches programming […] The post MIT just called for an educational revolution appeared first on LernerPython.

July and August are often when people take a break or go on vacation. But for me, this summer has been super busy, full of writing and improving LernerPython based […] The post AI seminars — just one of the big changes at LernerPython appeared first on LernerPython.

Using PyArrow dtypes in Python Pandas isn’t always faster: %timeit s_pyarr.mean() # 21.5ms%timeit s_np.mean() # 47.3ms %timeit s_pyarr.nlargest(10) # 667ms%timeit s_np.nlargest(10) # 663ms The post PyArrow dtypes in Pandas aren’t always faster than NumPy appeared first on LernerPython.

PyArrow strings in Python Pandas are smaller than Python strings. But they’re also far faster: %timeit s_pyarr.str.len() # 106 µs%timeit s_py.str.len() # 1.6 ms In many examples, PyArrow was far […] The post PyArrow strings in pandas: Faster and smaller than object dtype appeared first on LernerPyth...

Want PyArrow dtypes in your Python Pandas data frame? The dtypes are double[pyarrow], int64[pyarrow], and string[pyarrow], not the normal NumPy ones. Note: This is still experimental… but it’s also the […] The post PyArrow dtypes in pandas: Reading CSVs with dtype_backend appeared first on LernerPyt...

Using Python Pandas 3? Strings use PyArrow, not Pandas 2’s Python strings (dtype “object”): The post Pandas 3 string dtype: PyArrow strings cut memory use appeared first on LernerPython.

In Python Pandas 3, you can use PyArrow dtypes — which are nullable: What is s? 0 101 202 <NA> # pd.NA, not np.nan3 40dtype: int64[pyarrow] # see? Not float! The post PyArrow dtypes in Pandas 3: Missing values without float conversion appeared first on LernerPython.

Assign either np.nan or pd.NA to a Python Pandas series with a NumPy dtype, and it’ll be np.nan, a float. Which means the entire series has a dtype of float: […] The post Pandas missing values: Why assigning pd.NA gives you float64 appeared first on LernerPython.

If your dtype is too small, operations on your Python Pandas series will fail: Returns: 0 1101 1202 -126 # 🤯dtype: int8 This does give an error: s + 500OverflowError: […] The post Pandas int8 overflow: Why small dtypes wrap around silently appeared first on LernerPython.

Reading a CSV into a Python Pandas data frame? Use the “dtype” keyword arg and a dict to specify dtypes, and avoid the int64/float64/str defaults: The post Pandas read_csv dtype: Setting column types to save memory appeared first on LernerPython.

Want to change the dtype of a Python Pandas series? You can’t — at least, not by assigning: Instead, use “astype” to get a new series, and replace the old […] The post Changing a Pandas column dtype: Use astype, not assignment appeared first on LernerPython.

Get the dtype of a Python Pandas series with dtype: Get the dtypes of all columns in a data frame with “dtypes”, which returns a series: df.dtypes How many of […] The post Pandas dtypes: Checking column types with dtype and value_counts appeared first on LernerPython.

Want to get a quick look at a Python Pandas data frame you just created? Option 1, use df.head(n) to look at the first n rows: Option 2, use df.sample(n) […] The post Pandas data frame preview: df.head and df.sample appeared first on LernerPython.

If you want to get better at Pandas, the hard part isn’t finding tutorials. It’s finding problems worth solving. Most exercises hand you a tidy little table of five rows […] The post Free real-world Pandas exercises, with solutions appeared first on LernerPython.

Want to read a zipped CSV file into a Python Pandas data frame: Just pass the filename to read_csv: The file can contain a single CSV file, with an extension […] The post Pandas read_csv reads zipped CSV files without unzipping appeared first on LernerPython.

What sheets are in an Excel document you’re about to read into Python Pandas? You can check with: You’ll get a list of Python strings back. The post Excel sheet names in Pandas: Listing them with pd.ExcelFile appeared first on LernerPython.

Which is faster in Python Pandas, | or isin? I prefer isin; it’s clearer to write and read. Plus, fewer worries about parentheses. But is it faster? Depends on the […] The post Pandas isin vs | for filtering: which is faster appeared first on LernerPython.

Want rows in a Python Pandas data frame that might have several values? Another way (besides the | I showed yesterday) is the “isin” method: We’ll compare isin vs. | […] The post Pandas isin: Filtering rows that match multiple values appeared first on LernerPython.

Want rows in a Python Pandas data frame that might have several values? You can use the | operator, but be sure to use () to avoid precedence issues: The post Pandas OR conditions: Filtering rows with | and parentheses appeared first on LernerPython.

Stacking loc to filter a Python Pandas dataframe? Order can matter: The post Pandas loc filter order: Put the most selective filter first appeared first on LernerPython.

Filtering rows in a Python Pandas data frame? Use .loc: pd.col refers to the previous line’s returned data frame. So we can stack them: The post Filtering rows in Pandas with .loc and pd.col appeared first on LernerPython.

How much memory does a Python list use? sys.getsizeof will tell you, sort of: It reports the memory used by the list, but not its elements: The post Python list memory: Why sys.getsizeof ignores the elements appeared first on LernerPython.

Want stderr in your Python program to go somewhere else? Use redirect_stderr in contextlib: The post Redirecting stderr in Python: Using contextlib.redirect_stderr appeared first on LernerPython.

Writing a bunch to non-stdout in Python? You might want to use redirect_stdout from contextlib: The post redirect_stdout in Python: Sending print output to a file appeared first on LernerPython.

Want to print to stderr, not stdout, in a Python program? Use sys.stderr and the “file” kwarg: stdout and stderr often look identical in a terminal — but they aren’t! The post Python stderr: Printing with sys.stderr and the file kwarg appeared first on LernerPython.

Normally, “print” in Python writes to stdout. Write elsewhere with the “file” keyword argument: The post Python print to a file: Using the file keyword argument appeared first on LernerPython.

Want stdout from your Python program to go elsewhere? Assign sys.stdout to a writeable file. (Don’t forget to keep the original around!) Better: print’s file kwarg. The post Redirecting sys.stdout: Assigning print output to a file appeared first on LernerPython.