Pandas fillna() not working

Ryan

I am trying to replace NaN values ​​in a dataframe with the mean in the same row.

sample_df = pd.DataFrame({'A':[1.0,np.nan,5.0],
                   'B':[1.0,4.0,5.0],
                   'C':[1.0,1.0,4.0],
                   'D':[6.0,5.0,5.0],
                   'E':[1.0,1.0,4.0],
                   'F':[1.0,np.nan,4.0]})

sample_mean = sample_df.apply(lambda x: np.mean(x.dropna().values.tolist()) ,axis=1)

produce:

0    1.833333
1    2.750000
2    4.500000
dtype: float64

However, when I try to fillna()fill the missing dataframe values ​​with the values ​​from the series, it doesn't seem to work.

sample_df.fillna(sample_mean, inplace=True)

    A     B     C     D     E     F
0   1.0   1.0   1.0   6.0   1.0   1.0
1   NaN   4.0   1.0   5.0   1.0   NaN
2   5.0   5.0   4.0   5.0   4.0   4.0

What I expect is:

    A     B     C     D     E     F
0   1.0   1.0   1.0   6.0   1.0   1.0
1   2.75  4.0   1.0   5.0   1.0   2.75
2   5.0   5.0   4.0   5.0   4.0   4.0

I've reviewed other similar questions but can't seem to find the problem. Thanks in advance for your help.

BEN_YO

by usingpandas

sample_df.T.fillna(sample_df.T.mean()).T
Out[1284]: 
      A    B    C    D    E     F
0  1.00  1.0  1.0  6.0  1.0  1.00
1  2.75  4.0  1.0  5.0  1.0  2.75
2  5.00  5.0  4.0  5.0  4.0  4.00

Related

Pandas fillna() not working

Ryan I am trying to replace NaN values in a dataframe with the mean in the same row. sample_df = pd.DataFrame({'A':[1.0,np.nan,5.0], 'B':[1.0,4.0,5.0], 'C':[1.0,1.0,4.0], 'D':[6.0,5.0,5.0],

Pandas fillna() not working

Dor Hilman I'm trying to build a simple function to fill pandas columns of some distribution, but can't fill the entire table (df still has NaNs after fillna...) def simple_impute_missing(df): from numpy.random import normal rnd_filled = pd.DataFrame(

Pandas fillna() not working

Ryan I am trying to replace NaN values in a dataframe with the mean in the same row. sample_df = pd.DataFrame({'A':[1.0,np.nan,5.0], 'B':[1.0,4.0,5.0], 'C':[1.0,1.0,4.0], 'D':[6.0,5.0,5.0],

Python Pandas Fillna Median not working

Danielo I am trying to populate all nans in a dataframe with multiple columns and few rows. I'm using this to train a multivariate ML model, so I want to fill the nans of each column with the median. Just to test the median function, I do this: training_df.loc

Pandas fillna not working on subset of dataset

Hang Dabang I want to impute missing values df['box_office_revenue']with df['release_date'] == xand specified median df['genre'] == y. Here is my median finder function below. def find_median(df, year, genre, col_year, col_rev): median = df[(df[col_year] ==

Python Pandas Fillna Median not working

Danielo I am trying to populate all nans in a dataframe with multiple columns and few rows. I'm using this to train a multivariate ML model, so I want to fill the nans of each column with the median. Just to test the median function, I do this: training_df.loc

Pandas fillna not working on subset of dataset

Hang Dabang I want to impute missing values df['box_office_revenue']with df['release_date'] == xand specified median df['genre'] == y. Here is my median finder function below. def find_median(df, year, genre, col_year, col_rev): median = df[(df[col_year] ==

Pandas fillna not working on subset of dataset

Hang Dabang I want to impute missing values df['box_office_revenue']with df['release_date'] == xand specified median df['genre'] == y. Here is my median finder function below. def find_median(df, year, genre, col_year, col_rev): median = df[(df[col_year] ==

Pandas fillna not working on subset of dataset

Hang Dabang I want to impute missing values df['box_office_revenue']with df['release_date'] == xand specified median df['genre'] == y. Here is my median finder function below. def find_median(df, year, genre, col_year, col_rev): median = df[(df[col_year] ==

Pandas fillna not working on subset of dataset

Hang Dabang I want to impute missing values df['box_office_revenue']with df['release_date'] == xand specified median df['genre'] == y. Here is my median finder function below. def find_median(df, year, genre, col_year, col_rev): median = df[(df[col_year] ==

Python Pandas Fillna Median not working

Danielo I am trying to populate all nans in a dataframe with multiple columns and few rows. I'm using this to train a multivariate ML model, so I want to fill the nans of each column with the median. Just to test the median function, I do this: training_df.loc

Pandas fillna not working on subset of dataset

Hang Dabang I want to impute missing values df['box_office_revenue']with df['release_date'] == xand specified median df['genre'] == y. Here is my median finder function below. def find_median(df, year, genre, col_year, col_rev): median = df[(df[col_year] ==

Pandas fillna not working on subset of dataset

Hang Dabang I want to impute missing values df['box_office_revenue']with df['release_date'] == xand specified median df['genre'] == y. Here is my median finder function below. def find_median(df, year, genre, col_year, col_rev): median = df[(df[col_year] ==

Pandas fillna not working on subset of dataset

Hang Dabang I want to impute missing values df['box_office_revenue']with df['release_date'] == xand specified median df['genre'] == y. Here is my median finder function below. def find_median(df, year, genre, col_year, col_rev): median = df[(df[col_year] ==

Pandas fillna with lookup table

Zambi Having some trouble filling in NaNs. I want to have a dataframe column with several NaNs and populate them with values derived from a "lookup table" based on the values in another column. (You might recognize my data from the Titanic dataset)... Pcla

Pandas - fillna with row subset

Left__ I'm trying to fill certain rows with 0's where certain conditions apply. I'm trying now: df.loc[:,(df.Available == True) & (df.Intensity.isnull())].Intensity = df.loc[(df.Available == True) & (df.Intensity.isnull())].Intensity.fillna(0, inplace=True) T

Pandas fillna with lookup table

Zambi Having some trouble filling in NaNs. I want to have a dataframe column with several NaNs and populate them with values derived from a "lookup table" based on the values in another column. (You might recognize my data from the Titanic dataset)... Pcla

Pandas fillna empty dictionary

Sander I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic. fo

Conditional fillna() in pandas dataframe

RSM I have two dataframes below df1anddf2 df1： A B C D 1 Nora NaN Japan 2 Neo NaN India 3 Nord NaN Fuji 4 Noman 2020 Unknown df2： E F 1123 Neo 1124 Norm 1126 Nora I need to do a fillna once df1a

Is Pandas fillna wise?

Adaf Here is a simple example. d=pd.DataFrame({'x':[1,None,None,3,4],'y':[3,2,3,None,7],'z':[None,None,None,None,None]}) d['t']=d.mean(axis=1) Out[96]: x y z t 0 1.0 3.0 None 2.0 1 NaN 2.0 None 2.0 2 NaN 3.0 None 3.0 3 3.0 NaN N

Pandas fillna value increase

Eric M I have a dataframe with a column of consecutive but not adjacent numbers and missing values. I want to use the fillnafunction to fill missing values using the incremental value of the previous non-missing row. Here is a simplified table: index my_count

Pandas fillna with lookup table

Zambi Having some trouble filling in NaNs. I want to have a dataframe column with a few NaNs and populate them with values derived from a "lookup table" based on the values in another column. (You might recognize my data from the Titanic dataset)... Pclass

Pandas - fillna with row subset

Left__ I'm trying to fill certain rows with 0's where certain conditions apply. I'm trying now: df.loc[:,(df.Available == True) & (df.Intensity.isnull())].Intensity = df.loc[(df.Available == True) & (df.Intensity.isnull())].Intensity.fillna(0, inplace=True) T

Pandas fillna using groupby

niche I'm trying to estimate values using rows with similar column values. For example, I have this dataframe one | two | three 1 1 10 1 1 nan 1 1 nan 1 2 nan 1 2 20 1 2 nan 1 3 nan 1 3 na

Is Pandas fillna wise?

Adaf Here is a simple example. d=pd.DataFrame({'x':[1,None,None,3,4],'y':[3,2,3,None,7],'z':[None,None,None,None,None]}) d['t']=d.mean(axis=1) Out[96]: x y z t 0 1.0 3.0 None 2.0 1 NaN 2.0 None 2.0 2 NaN 3.0 None 3.0 3 3.0 NaN N

Pandas - fillna with row subset

Left__ I'm trying to fill certain rows with 0's where certain conditions apply. I'm trying now: df.loc[:,(df.Available == True) & (df.Intensity.isnull())].Intensity = df.loc[(df.Available == True) & (df.Intensity.isnull())].Intensity.fillna(0, inplace=True) T

Conditional fillna() in pandas dataframe

RSM I have two dataframes below df1anddf2 df1： A B C D 1 Nora NaN Japan 2 Neo NaN India 3 Nord NaN Fuji 4 Noman 2020 Unknown df2： E F 1123 Neo 1124 Norm 1126 Nora I need to do a fillna once df1a

Pandas fillna empty dictionary

Sander I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic. fo

Pandas fillna using groupby

niche I'm trying to estimate values using rows with similar column values. For example, I have this dataframe one | two | three 1 1 10 1 1 nan 1 1 nan 1 2 nan 1 2 20 1 2 nan 1 3 nan 1 3 na