DataFrames, vectorization, joins, groupby, plotting
Practise these as flashcards. One or two lines each; the ones you miss come back first.
Start freeA NumPy array is a fixed-type, multidimensional array that supports fast, vectorized operations. It's preferred over Python lists because it's much faster and uses less memory, especially for large numerical datasets.
You can select a column using either df['column_name'] or df.column_name. Both return a Series, but bracket notation is more robust, especially if the column name has spaces.
.loc accessor do in pandas?.loc is used for label-based indexing, allowing you to select rows and columns by their names rather than their integer positions.
You can use df.isnull() to get a boolean mask of missing values, or df.isnull().sum() to count them per column.
.loc and .iloc in pandas..loc accesses rows and columns by their labels, while .iloc uses integer positions. For example, .loc['row1', 'colA'] vs .iloc[0, 1]. Using the wrong accessor can lead to errors or unexpected results.
I'd use df.groupby('region')['sales'].sum() to group the data by region and sum the sales for each group. This provides a quick summary of sales performance across regions.
Common pitfalls include mismatched join keys, duplicate column names, and unintended Cartesian products if keys aren't unique. It's important to check key uniqueness and column overlaps before merging.
I inspect missing value patterns, then either fill them with statistical values like mean or median, or drop rows/columns with too many missing values, depending on the context and impact on the data.
.apply with custom functions discouraged for large DataFrames?.apply is slow because it operates row-by-row in Python, whereas vectorized operations run in compiled code and are much faster. For large DataFrames, vectorization is crucial for performance.
Optimizing dtypes, such as converting object columns to category or using smaller integer types, can significantly reduce memory usage. Also, dropping unnecessary columns helps.
A pivot table summarizes data by grouping and aggregating values across categories, similar to Excel. It's useful for reshaping data and quickly comparing metrics across multiple dimensions.
By expressing the calculation as an array operation, such as result = a * b instead of looping through elements, NumPy leverages optimized C code for much faster execution.