"fillna" command in python not returning mean using pandas

indiana writer

I am trying to run fillna command in python. It just can't replace Nan values ​​with anything and doesn't return an error.

import pandas as pd
import io
import requests
import numpy as np
url='https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.data'
s=requests.get(url).content
df=pd.read_csv(io.StringIO(s.decode('utf-8')))
df.columns=['Scn', 'A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'A10', 'CLASS']


df.to_csv("wisconsinbreast.csv")

m,n=df.shape
#print(m,n)
df = df.replace('?', np.nan)
#print(df)
#print(df.mean()) 
print(df.fillna(df.mean()))

On line 22, Nan is still there. I've found everything I can find by searching for the problem here, but that doesn't even give me feedback on why it's failing. From what I understand, df.mean should be calculated without Nan values, but df.mean doesn't return any values ​​for columns that contain Nan.

pirate

na_valuesexistread_csv

That '?'would crash everything. When read_csvit sees it, it assumes the entire column is of type D object, and reads it as a string. Of course, you can fix this after the fact, but I recommend using a parameter to fix it na_valuesat the beginning :

df = pd.read_csv(io.StringIO(s.decode('utf-8')), na_values=['?'])

pd.to_numeric

But if you really want to fix it after the fact, do this insteadreplace

df.A7 = pd.to_numeric(df.A7, errors='coerce')

In either case, it fillnashould work as expected

df.fillna(df.mean())

         Scn  A2  A3  A4  A5  A6         A7  A8  A9  A10  CLASS
0    1002945   5   4   4   5   7  10.000000   3   2    1      2
1    1015425   3   1   1   1   2   2.000000   3   1    1      2
2    1016277   6   8   8   1   3   4.000000   3   7    1      2
3    1017023   4   1   1   3   2   1.000000   3   1    1      2
4    1017122   8  10  10   8   7  10.000000   9   7    1      4
5    1018099   1   1   1   1   2  10.000000   3   1    1      2
6    1018561   2   1   2   1   2   1.000000   3   1    1      2
7    1033078   2   1   1   1   2   1.000000   1   1    5      2
8    1033078   4   2   1   1   2   1.000000   2   1    1      2
9    1035283   1   1   1   1   1   1.000000   3   1    1      2
10   1036172   2   1   1   1   2   1.000000   2   1    1      2
11   1041801   5   3   3   3   2   3.000000   4   4    1      4
12   1043999   1   1   1   1   2   3.000000   3   1    1      2
13   1044572   8   7   5  10   7   9.000000   5   5    4      4
14   1047630   7   4   6   4   6   1.000000   4   3    1      4
15   1048672   4   1   1   1   2   1.000000   2   1    1      2
16   1049815   4   1   1   1   2   1.000000   3   1    1      2
17   1050670  10   7   7   6   4  10.000000   4   1    2      4
18   1050718   6   1   1   1   2   1.000000   3   1    1      2
19   1054590   7   3   2  10   5  10.000000   5   4    4      4
20   1054593  10   5   5   3   6   7.000000   7  10    1      4
21   1056784   3   1   1   1   2   1.000000   2   1    1      2
22   1057013   8   4   5   1   2   3.548387   7   3    1      4
23   1059552   1   1   1   1   2   1.000000   3   1    1      2
24   1065726   5   2   3   4   2   7.000000   3   6    1      4

Related

"fillna" command in python not returning mean using pandas

indiana writer I am trying to run fillna command in python. It just can't replace Nan values with anything and doesn't return an error. import pandas as pd import io import requests import numpy as np url='https://archive.ics.uci.edu/ml/machine-learning-databa

Multicolumn Pandas fillna with mean and groupby

DataGwynn I'm trying to group by and populate multiple columns at the same time. I'm attaching a picture of what the data looks like and the code I'm having trouble with. This is sample data I created to reflect the actual data as it is confidential. There are

Fillna Pandas NaN Mean and Median

User HG I'm starting with python and data science and I have a .csv file with over 5000 rows. I would like to replace the "experienced NaN" values with the mean for data scientists and the median for data engineers. How to group it and use fillna. Everytime I

Multicolumn Pandas fillna with mean and groupby

DataGwynn I'm trying to group by and populate multiple columns at the same time. I'm attaching a picture of what the data looks like and the code I'm having trouble with. This is sample data I created to reflect the actual data as it is confidential. There are

Multicolumn Pandas fillna with mean and groupby

DataGwynn I'm trying to group by and populate multiple columns at the same time. I'm attaching a picture of what the data looks like and the code I'm having trouble with. This is sample data I created to reflect the actual data as it is confidential. There are

Fillna Pandas NaN Mean and Median

User HG I'm starting with python and data science and I have a .csv file with over 5000 rows. I would like to replace the "experienced NaN" values with the mean for data scientists and the median for data engineers. How to group it and use fillna. Everytime I

Multicolumn Pandas fillna with mean and groupby

DataGwynn I'm trying to group by and populate multiple columns at the same time. I'm attaching a picture of what the data looks like and the code I'm having trouble with. This is sample data I created to reflect the actual data as it is confidential. There are

Multicolumn Pandas fillna with mean and groupby

DataGwynn I'm trying to group by and populate multiple columns at the same time. I'm attaching a picture of what the data looks like and the code I'm having trouble with. This is sample data I created to reflect the actual data as it is confidential. There are

Fillna Pandas NaN Mean and Median

User HG I'm starting with python and data science and I have a .csv file with over 5000 rows. I would like to replace the "experienced NaN" values with the mean for data scientists and the median for data engineers. How to group it and use fillna. Everytime I

Pandas fillna using groupby

niche I'm trying to estimate values using rows with similar column values. For example, I have this dataframe one | two | three 1 1 10 1 1 nan 1 1 nan 1 2 nan 1 2 20 1 2 nan 1 3 nan 1 3 na

Pandas fillna using groupby

niche I'm trying to estimate values using rows with similar column values. For example, I have this dataframe one | two | three 1 1 10 1 1 nan 1 1 nan 1 2 nan 1 2 20 1 2 nan 1 3 nan 1 3 na

Using fillna with conditional pandas

Josepha Shelkens I have the following dataframe and I want to apply the padding in the following way : data: print(for_stack.to_dict()) {2.0: {'A_cj8e134xu02pixvky4r70o0se': 1.0, 'A_cj8t63fsb04ga5bm4ongrlx6h': 1.0}, 3.0: {'A_cj8e134xu02pixvky4r70o0se': 2.0, '

Pandas fillna using groupby

niche I'm trying to estimate values using rows with similar column values. For example, I have this dataframe one | two | three 1 1 10 1 1 nan 1 1 nan 1 2 nan 1 2 20 1 2 nan 1 3 nan 1 3 na

Pandas fillna using groupby

niche I'm trying to estimate values using rows with similar column values. For example, I have this dataframe one | two | three 1 1 10 1 1 nan 1 1 nan 1 2 nan 1 2 20 1 2 nan 1 3 nan 1 3 na

Using fillna with conditional pandas

Josepha Shelkens I have the following dataframe and I want to apply the padding in the following way : data: print(for_stack.to_dict()) {2.0: {'A_cj8e134xu02pixvky4r70o0se': 1.0, 'A_cj8t63fsb04ga5bm4ongrlx6h': 1.0}, 3.0: {'A_cj8e134xu02pixvky4r70o0se': 2.0, '

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f