What Data Scientists Forget About Memory
A guide for data analysts and scientists to write more robust, efficient, and predictable code by understanding how memory really works.
Introduction: The Bugs That Aren’t in Your Logic
You have a Python script that runs perfectly on a small sample of data but crashes with a cryptic MemoryError on the full dataset. Or maybe you’ve encountered a bug in a data pipeline that vanishes when you try to debug it step-by-step. These frustrating problems often have nothing to do with a flaw in your business logic and everything to do with a gap in understanding what the computer is doing “under the hood.”
Many of these high-level, unpredictable issues have their roots in fundamental computer science concepts about memory. Understanding them isn’t just an academic exercise; it’s a practical skill for writing better, more reliable code. This post is a synthesis based on the CS50x 2025 (Harvard) lecture on Memory, available on YouTube.
Takeaway 1: “String” is an Illusion
In many introductory contexts, a string is presented as a basic data type. The CS50 library, for instance, uses it as a helpful “training wheel” to get students started. However, in a foundational language like C, there is no built-in string data type. It’s an abstraction.
What we call a string is actually a char *—a pointer to a character. This pointer is simply the memory address of the first character in a sequence. To know where the string ends, C relies on a special sentinel value: the null terminator (\0), a byte containing all zeros. Functions like printf read from the starting address character by character until they hit this null terminator. Without it, the function would just keep reading into adjacent memory, causing unpredictable behavior.
Data Science Connection: This concept is the ancestor of how high-level languages like Python and R manage strings and other variable-length data. It explains why string manipulation can be computationally expensive; the system is often scanning for a termination point or managing underlying arrays. For data scientists, this is crucial when parsing raw text files or working with APIs; a missing or incorrect terminator byte can lead to corrupted data or parsing failures, like a log file line being read incorrectly and truncating important error information.
Takeaway 2: Why == Can Lie About Equality
When you compare two integer variables in C with ==, you’re comparing their values. But if you compare two string variables with ==, you are not comparing their content. Instead, you are comparing their memory addresses.
If you have two identical strings, “high” and “high”, but they are stored in different locations in memory, the == operator will return false. Their content is the same, but their addresses are different. This is why C requires a special function like strcmp (string comparison), which performs a character-by-character check to determine if the strings’ values are truly identical.
Data Science Connection: This is a direct parallel to the concept of object identity versus value equality in Python. Using is in Python compares memory addresses (identity), while == typically compares values (equality). For complex objects like Pandas DataFrames, even == can be ambiguous. This is why dedicated methods like pandas.DataFrame.equals() exist. The fundamental C behavior explains why you can’t just compare two DataFrames with a simple operator; you need a robust method that iterates through and compares their actual contents, just like strcmp does for C strings.
Takeaway 3: A Copy Might Just Be an Alias
In C, an instruction like string t = s; might look like it’s creating a new copy of string s and naming it t. It is not. This operation does not copy the string’s content (the characters ‘H’, ‘I’, ‘!’, ‘\0’). It only copies the address of the string’s first character.
This means both variables, s and t, now point to the exact same block of memory. They are two different names for the same underlying data. Consequently, if you modify the string using the variable t (for example, by capitalizing the first letter), you are also changing the string for s. There is only one string, and both variables point to it.
Data Science Connection: This is the root of the “view vs. copy” problem that frequently confuses Pandas users. A data scientist might filter a DataFrame, assign it to a new variable, and start making changes, believing they are working on an independent copy. But if Pandas returned a “view” (an alias pointing to the original data), they are unintentionally altering their source dataset. This can lead to subtle, dangerous bugs that are hard to trace, as the original data is “mysteriously” modified by an operation performed somewhere else.
“When you use literally equals equals to compare the two strings… that is compare the two Char stars that is compare the two addresses clearly they’re in different places… we needed a solution for this and that is why we introduced stir comp the function that actually addressed this properly.” — Harvard CS50x 2025 (YouTube) — Lecture: Memory
Takeaway 4: Memory Is a Finite Resource You Must Manage
When a program needs memory while it’s running—for example, to store data read from a file—it can request it from the operating system. In C, this is done with a function called malloc() (memory allocation), which asks for a specific number of bytes from a large memory pool called the “heap.”
However, this request can fail. If the system is out of available memory, malloc() will return NULL (a special pointer to address zero) to signal failure. Robust code must always check for this possibility. Furthermore, once the program is finished with that memory, it must explicitly return it to the system using free(). Failing to do so creates a “memory leak,” where the program holds onto memory it no longer needs, potentially causing it to run out of resources and crash over time.
Data Science Connection: This is directly applicable to working with large datasets. Attempting to load a massive CSV or Parquet file into a Pandas DataFrame can easily cause a script to crash if it requests more memory than the system has available. The principles of malloc and free are fundamental to writing scalable data pipelines, understanding why certain processes are “memory-bound,” and appreciating the design of modern data tools that use techniques like chunking or memory mapping to process data that won’t fit into memory all at once.
Takeaway 5: Functions Have Private Workspaces (The Stack)
When a function is called, it gets its own private workspace in a region of memory called the “stack.” This “stack frame” holds its local variables, including the arguments passed to it.
The CS50 lecture demonstrates this with a swap function intended to swap two integer values. The initial version fails because it only swaps copies of the values within its own private stack frame. This is known as “pass by value.” Once the function finishes, its stack frame disappears, and the original variables in the calling function (main) are left completely unchanged. The corrected version works by “pass by reference”—passing the memory addresses of the original variables. This gives the swap function the power to reach back into the caller’s stack frame and modify the original variables directly.
Data Science Connection: This concept is critical for understanding variable scope and mutability when writing data transformation functions. It helps explain why a function might not modify a DataFrame passed to it as expected (e.g., if it reassigns the variable internally instead of modifying the object in place). It also underscores the design principles of “pure functions”—functions that don’t modify their inputs—which are essential for creating reproducible, testable, and bug-free analysis pipelines.
Takeaway 6: The Billion-Dollar Bug Is an Off-by-One Error
A “buffer overflow” occurs when a program tries to write data past the allocated boundaries of a memory buffer, like an array. It’s like trying to pour a gallon of water into a pint glass—the excess has to go somewhere, and it often overwrites adjacent memory, causing corruption and crashes.
The lecture references a real-world incident involving CrowdStrike security software. A bug in an update caused systems worldwide to fail. The root cause was astonishingly simple: the code expected an array with 20 values but received 21. The attempt to access the non-existent 21st element produced an “out-of-bounds memory read,” leading to a system crash. C provides no automatic protection against this, making it a powerful demonstration of a dangerous class of bug.
Data Science Connection: This is the ultimate lesson in defensive programming for data work. When parsing files, iterating over lists, or processing API responses, you cannot trust that the incoming data conforms to your expectations. This low-level vulnerability highlights the high-level importance of robust error handling and input validation. Always add explicit checks for expected formats, sizes, and boundaries before processing data to prevent your entire pipeline from crashing due to one unexpected value.
What to Apply Today: A Practical Checklist
- Interrogate Your Equality Checks: When comparing complex objects like models or DataFrames, pause and confirm if you need to know if they are the exact same object in memory (identity) or if they simply contain the same data (equality). Use the appropriate method (
isvs..equals()) for your goal. - Audit Your DataFrame Modifications: Before modifying what you think is a copy of a DataFrame, explicitly check if it’s a view or a copy. A simple change on a view can silently corrupt your original dataset. Use
.copy()to ensure you have an independent object. - Be Proactive About Memory: When working with large files, don’t just load them and hope for the best. Use tools to check available system memory, and process files in smaller chunks (e.g., using
chunksizeinpandas.read_csv) to avoid requesting more memory than is available. - Validate Data Boundaries: Whenever you access data by an index, such as in a list or array, consider it a potential point of failure. Add checks to ensure the index is within the expected range, especially when the size of the data is determined by an external source.
- Never Trust Input Data: Treat data from files, APIs, or user input as fundamentally unreliable. Before performing operations, add assertions or explicit checks for data types, sizes, and formats to catch unexpected issues before they corrupt your analysis.
9. Closing Thoughts
These low-level memory concepts are not just academic trivia; they directly surface as the bugs, performance bottlenecks, and security risks we encounter in high-level data science work. By understanding what happens in memory, we can move from being confused by symptoms to diagnosing the root cause. As the CS50 course progresses, these same principles become the building blocks for creating the efficient data structures that power our databases and algorithms.
What recent ‘weird’ bug in your code could now be explained by one of these core memory principles?
