Python pandas fillna()


Dashan Jadhav

I have a large dataframe with the following values:

Name  A  B  C  D  E  F  G # Coulmns
Matt  1  n  n  n  5  n  5  # rows
Jake  n  n  2  n  3  n  n
Paul  2  n  3  n  n  8  n

I just want to fill NA values ​​with previous values:

df.fillna(method='ffill')

This is the forward fill method which gives me the following output:

Name  A  B  C  D  E  F  G # Coulmns
Matt  1  1  1  1  5  5  5  # rows
Jake  n  n  2  2  3  3  3
Paul  2  2  3  3  3  8  8

The problem is that I don't want it to go to the end and stop if there are no more values ​​behind. See line 2, Jake, it must stop at E because it doesn't have any value in the last column. Desired output:

Name  A  B  C  D  E  F  G # Coulmns
Matt  1  1  1  1  5  5  5  # rows
Jake  n  n  2  2  3  n  n
Paul  2  2  3  3  8  8  n

Is there anyway to do this with pandas?

code:

 for i in range(len(df)):
    df.loc[i][1:].fillna(method='ffill',inplace=True)
Jesler

I think you need:

  • set_indexby first column
  • Replace s with NaNbackfill and NaNuseisnull()
  • Replace NaN with forward padding and replace NaNs withmask
  • last column indexreset_index

df = df.set_index('Name')
m = df.bfill(axis=1).isnull()
df = df.ffill(axis=1).mask(m)
print (df)
    Name    A    B    C    D    E    F    G
0    Matt  1.0  1.0  1.0  1.0  5.0  5.0  5.0
1    Jake  NaN  NaN  2.0  2.0  3.0  NaN  NaN
2    Paul  2.0  2.0  3.0  3.0  3.0  8.0  NaN

Related


Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python pandas fillna()

Dashan Jadhav I have a large dataframe with the following values: Name A B C D E F G # Coulmns Matt 1 n n n 5 n 5 # rows Jake n n 2 n 3 n n Paul 2 n 3 n n 8 n I just want to fill NA values with previous values: df.fillna(method='f

Python Pandas Fillna Median not working

Danielo I am trying to populate all nans in a dataframe with multiple columns and few rows. I'm using this to train a multivariate ML model, so I want to fill the nans of each column with the median. Just to test the median function, I do this: training_df.loc

Fillna in multiple columns in Python Pandas

ozzy: I have a pandas dataFrame of mixed types, some are strings and some are numbers. I want to replace NAN values in string column with "." and NAN values in float column with 0. Consider this small dummy example: df = pd.DataFrame({'Name':['Jack','Sue',pd.n

Python Pandas Fillna Median not working

Danielo I am trying to populate all nans in a dataframe with multiple columns and few rows. I'm using this to train a multivariate ML model, so I want to fill the nans of each column with the median. Just to test the median function, I do this: training_df.loc

Fillna in multiple columns in Python Pandas

ozzy: I have a pandas dataFrame of mixed types, some are strings and some are numbers. I want to replace NAN values in string column with "." and NAN values in float column with 0. Consider this small dummy example: df = pd.DataFrame({'Name':['Jack','Sue',pd.n

Python Pandas Fillna Median not working

Danielo I am trying to populate all nans in a dataframe with multiple columns and few rows. I'm using this to train a multivariate ML model, so I want to fill the nans of each column with the median. Just to test the median function, I do this: training_df.loc

"fillna" command in python not returning mean using pandas

indiana writer I am trying to run fillna command in python. It just can't replace Nan values with anything and doesn't return an error. import pandas as pd import io import requests import numpy as np url='https://archive.ics.uci.edu/ml/machine-learning-databa

Pandas .fillna() not filling values in DataFrame in Python 3

central analyst I am running Pandas in Python 3 and found the following: import pandas as pd import numpy as np from pandas import DataFrame from numpy import nan df = DataFrame([[1, nan], [nan, 4], [5, 6]]) print(df) df2 = df df2.fillna(0) print(df2) ret

Pandas .fillna() not filling values in DataFrame in Python 3

central analyst I am running Pandas in Python 3 and found the following: import pandas as pd import numpy as np from pandas import DataFrame from numpy import nan df = DataFrame([[1, nan], [nan, 4], [5, 6]]) print(df) df2 = df df2.fillna(0) print(df2) ret

Problems with NaN values and fillna function python pandas

this I ran into a problem and hope someone has a good solution. I am reading an Excel file. I use keep_default_na=False because there is a product name called "NA" and I don't want pandas to change it to NaN. df = pd.read_excel('Import_orders.xlsx', keep_defau

"fillna" command in python not returning mean using pandas

indiana writer I am trying to run fillna command in python. It just can't replace Nan values with anything and doesn't return an error. import pandas as pd import io import requests import numpy as np url='https://archive.ics.uci.edu/ml/machine-learning-databa

Python Pandas Fillna only one row with specific value

Lagos edit: I have (not very simple) a dataframe: df = pd.DataFrame([1, 2, np.nan, np.nan, np.nan, np.nan, 3, 4 , np.nan, np.nan, np.nan, 5], columns=['att1']) att1 0 1.0000 1 2.0000 2 nan 3 nan 4 nan 5 nan 6 3.0000 7 4.0000 8