Is Pandas fillna wise?

Adaf

Here is a simple example.

d=pd.DataFrame({'x':[1,None,None,3,4],'y':[3,2,3,None,7],'z':[None,None,None,None,None]})
d['t']=d.mean(axis=1)

Out[96]: 
     x    y     z    t
0  1.0  3.0  None  2.0
1  NaN  2.0  None  2.0
2  NaN  3.0  None  3.0
3  3.0  NaN  None  3.0
4  4.0  7.0  None  5.5

I want to fill the NA with the mean I am adding each row to the row.The mean
is t. Turns out what I want is:

x y z 
1 3 2
2 2 2
3 3 3
3 3 3
4 7 5.5
Clement Kleisha

You can do this:

d.T.fillna(d.T.mean(axis=0), axis=0).T

You have to transpose the dataframe because:

d.fillna(d.mean(axis=1), axis=1)

Not implemented (NotImplementedError)

Related

Is Pandas fillna wise?

Adaf Here is a simple example. d=pd.DataFrame({'x':[1,None,None,3,4],'y':[3,2,3,None,7],'z':[None,None,None,None,None]}) d['t']=d.mean(axis=1) Out[96]: x y z t 0 1.0 3.0 None 2.0 1 NaN 2.0 None 2.0 2 NaN 3.0 None 3.0 3 3.0 NaN N

Pandas fillna with lookup table

Zambi Having some trouble filling in NaNs. I want to have a dataframe column with several NaNs and populate them with values derived from a "lookup table" based on the values in another column. (You might recognize my data from the Titanic dataset)... Pcla

Pandas - fillna with row subset

Left__ I'm trying to fill certain rows with 0's where certain conditions apply. I'm trying now: df.loc[:,(df.Available == True) & (df.Intensity.isnull())].Intensity = df.loc[(df.Available == True) & (df.Intensity.isnull())].Intensity.fillna(0, inplace=True) T

Pandas fillna() not working

Ryan I am trying to replace NaN values in a dataframe with the mean in the same row. sample_df = pd.DataFrame({'A':[1.0,np.nan,5.0], 'B':[1.0,4.0,5.0], 'C':[1.0,1.0,4.0], 'D':[6.0,5.0,5.0],

Pandas fillna with lookup table

Zambi Having some trouble filling in NaNs. I want to have a dataframe column with several NaNs and populate them with values derived from a "lookup table" based on the values in another column. (You might recognize my data from the Titanic dataset)... Pcla

Pandas fillna empty dictionary

Sander I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic. fo

Conditional fillna() in pandas dataframe

RSM I have two dataframes below df1anddf2 df1： A B C D 1 Nora NaN Japan 2 Neo NaN India 3 Nord NaN Fuji 4 Noman 2020 Unknown df2： E F 1123 Neo 1124 Norm 1126 Nora I need to do a fillna once df1a

Pandas fillna value increase

Eric M I have a dataframe with a column of consecutive but not adjacent numbers and missing values. I want to use the fillnafunction to fill missing values using the incremental value of the previous non-missing row. Here is a simplified table: index my_count

Pandas fillna with lookup table

Zambi Having some trouble filling in NaNs. I want to have a dataframe column with a few NaNs and populate them with values derived from a "lookup table" based on the values in another column. (You might recognize my data from the Titanic dataset)... Pclass

Pandas - fillna with row subset

Left__ I'm trying to fill certain rows with 0's where certain conditions apply. I'm trying now: df.loc[:,(df.Available == True) & (df.Intensity.isnull())].Intensity = df.loc[(df.Available == True) & (df.Intensity.isnull())].Intensity.fillna(0, inplace=True) T

Pandas fillna using groupby

niche I'm trying to estimate values using rows with similar column values. For example, I have this dataframe one | two | three 1 1 10 1 1 nan 1 1 nan 1 2 nan 1 2 20 1 2 nan 1 3 nan 1 3 na

Pandas - fillna with row subset

Left__ I'm trying to fill certain rows with 0's where certain conditions apply. I'm trying now: df.loc[:,(df.Available == True) & (df.Intensity.isnull())].Intensity = df.loc[(df.Available == True) & (df.Intensity.isnull())].Intensity.fillna(0, inplace=True) T

Conditional fillna() in pandas dataframe

RSM I have two dataframes below df1anddf2 df1： A B C D 1 Nora NaN Japan 2 Neo NaN India 3 Nord NaN Fuji 4 Noman 2020 Unknown df2： E F 1123 Neo 1124 Norm 1126 Nora I need to do a fillna once df1a

Pandas fillna() not working

Ryan I am trying to replace NaN values in a dataframe with the mean in the same row. sample_df = pd.DataFrame({'A':[1.0,np.nan,5.0], 'B':[1.0,4.0,5.0], 'C':[1.0,1.0,4.0], 'D':[6.0,5.0,5.0],

Pandas fillna empty dictionary

Sander I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic. fo

Pandas fillna using groupby

niche I'm trying to estimate values using rows with similar column values. For example, I have this dataframe one | two | three 1 1 10 1 1 nan 1 1 nan 1 2 nan 1 2 20 1 2 nan 1 3 nan 1 3 na

Pandas fillna empty dictionary

Sander I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic. fo

Using fillna with conditional pandas

Josepha Shelkens I have the following dataframe and I want to apply the padding in the following way : data: print(for_stack.to_dict()) {2.0: {'A_cj8e134xu02pixvky4r70o0se': 1.0, 'A_cj8t63fsb04ga5bm4ongrlx6h': 1.0}, 3.0: {'A_cj8e134xu02pixvky4r70o0se': 2.0, '

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Pandas slicing and indexing with fillna

Vikash B I have a pandas dataframe tdf and I am extracting a slice based on boolean labels idx = tdf['MYcol1'] == 1 myslice = tdf.loc[idx] //I want myslice to be a view not a copy Now I want to fill in the missing values in a column of myslice , I want this t

Pandas fillna with list/array

Rutger Kassies Is there a convenient way to populate na values with the (first) value of an array or column? Imagine the following DataFrame: dfcolors = pd.DataFrame({'Colors': ['Blue', 'Red', np.nan, 'Green', np.nan, np.nan, 'Brown']}) Colors 0 Blue 1

pandas fillna on datetime object

Luke I am trying to run fillnaon a column of type datetime64[ns] . When I run something like:df['date'].fillna(datetime("2000-01-01")) I get:TypeError: an integer is required Can it be solved? Jeff This should work in 0.12 and 0.13 (release only). @DSM pointed

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f