3 Data Preprocessing and Feature Extraction II: Smart Binarization (3 pts)

In the previous section, we discussed the naive binarization of the Income dataset. While this method can be

effective for certain machine learning models, there are more refined data preprocessing techniques that can yield

better results, especially when dealing with a mix of numerical and categorical data. This section delves into

smarter binarization methods, particularly focusing on differential handling of numerical and categorical features

and scaling the numerical data for better algorithmic performance.

1. For machine algorithms sensitive to distance metrics such as k-NN, it's pivotal to preprocess data more intelli-

gently. In our case, we want to treat numerical (age and hours-per-week) and categorical fields differently.

For categorical data, we continue to use OneHotEncoder for binarization. However, for numerical data, instead

of binarizing them directly, we let them remain in their original format. Here's how you'd typically proceed:

Differentiate between numerical and categorical fields in your dataset (i.e., identify their column names).

• Apply OneHotEncoder only to the categorical fields (in our case, all but age and hours-per-week).

• Combine the processed categorical data with the numerical data to form the modified dataset.

Here, we provide an example for the toy dataset:

• Import the necessary libraries:

from sklearn.compose import Column Transformer

from sklearn.preprocessing import OneHotEncoder

Instantiate the preprocessing methods for numerical and categorical data:

num_processor = 'passthrough' # i.e., no transformation

cat_processor OneHotEncoder (sparse=False, handle_unknown=':

=

Fit the preprocessor to the dataset and transform:

preprocessor = ColumnTransformer ( [

])

('num', num_processor, ['age']),

('cat', cat_processor, ['sector'])

preprocessor.fit (data)

processed_data = preprocessor.transform (data)

And the processed_data will give us:

='ignore')/n[[45. 1. 0. 0.]

[33. 0. 0. 1.]

[19. 0. 0. 1.]

[47. 0. 0. 1.]

[30. 0. 1. 0.]

[37. 0. 0. 1.]

[35. 0. 0. 1.]

[33. 0. 0. 1.]

[40. 0. 0. 1.]

[54. 1. 0. 0.]]

You can also check the name of each feature by preprocessor.get_feature_names_out (), which gives us:

['num__age' 'cat__sector_Federal-gov' 'cat__sector_Local-gov' 'cat__sector_Private']

After this preprocessing step, make your new number of features is around 90, as in Part 1 Q5.

Question: Re-execute all experiments with varying values of k (Part 2, Q 4a) and report the new results. Do

you notice any performance improvements compared to the initial results? If so, why? If not, why do you think

that is? (1.5 pts) (Hint: 1-NN dev error should be ~27% and the best dev error should be ~21%)