Pandas fillna empty dictionary

Sander

I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values ​​are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic.

for adding columns

tspd['metadata'] = {} # fails
tspd['metadata'] = [{} for _ in tspd.index] # works

used to fill in missing values

tspd['metadata'].replace(np.nan,{}) # does nothing
tspd['metadata'].fillna({})  # likewise does nothing
tspd.loc[tspd['metadata'].isna(), 'metadata'] = {} # error
tspd['metadata'] = tspd['metadata'].where(~tspd['metadata'].isna(), other={}) # this sets the NaN values to <built-in method values of dict object>

So adding that column works, but it's a bit cumbersome. It seems that the values ​​cannot be replaced without some (slow) looping.

Jesler

You can use np.nan == np.nanis False, so missing values ​​can be replaced:

tspd = pd.DataFrame({'a': [0,1,2], 'metadata':[{'a':'s'}, np.nan, {'d':'e'}]})

tspd['metadata'] = tspd['metadata'].apply(lambda x: {} if x != x else x)
print(tspd)

   a    metadata
0  0  {'a': 's'}
1  1          {}
2  2  {'d': 'e'}

either:

tspd['metadata'] = [{} if x != x else x for x in tspd['metadata']]

Related

Pandas fillna empty dictionary

Sander I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic. fo

Pandas fillna empty dictionary

Sander I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic. fo

Pandas fillna empty dictionary

Sander I have a pandas dataframe with a column "metadata" which should contain a dictionary as values. However, some values are missing and set to NaN. I want to change to {}. Sometimes the whole column is lost and initializing it to {} is also problematic. fo

Fill a list with empty values in Pandas using fillna

Fragments Given one pd.Series, I want to replace empty values with a list. That is, given: import numpy as np import pandas as pd ser = pd.Series([0,1,np.nan]) I want a function that can return 0 0 1 1 2 [nan] However, if I try to use the na

.fillna column (if two cells are empty in pandas)

truthling Can someone tell me why in my for loop df_all = pd.read_csv("assembly_summary.txt", delimiter='\t', index_col=0) for row in df_all.index: if pd.isnull(df_all.infraspecific_name[row]) and pd.isnull(df_all.isolate[row]): df_all.infraspecifi

Fill a list with empty values in Pandas using fillna

Fragments Given one pd.Series, I want to replace empty values with a list. That is, given: import numpy as np import pandas as pd ser = pd.Series([0,1,np.nan]) I want a function that can return 0 0 1 1 2 [nan] However, if I try to use the na

Fill a list with empty values in Pandas using fillna

Fragments Given one pd.Series, I want to replace empty values with a list. That is, given: import numpy as np import pandas as pd ser = pd.Series([0,1,np.nan]) I want a function that can return 0 0 1 1 2 [nan] However, if I try to use the na

Fill a list with empty values in Pandas using fillna

Fragments Given one pd.Series, I want to replace empty values with a list. That is, given: import numpy as np import pandas as pd ser = pd.Series([0,1,np.nan]) I want a function that can return 0 0 1 1 2 [nan] However, if I try to use the na

Fill a list with empty values in Pandas using fillna

Fragments Given one pd.Series, I want to replace empty values with a list. That is, given: import numpy as np import pandas as pd ser = pd.Series([0,1,np.nan]) I want a function that can return 0 0 1 1 2 [nan] However, if I try to use the na

Fill a list with empty values in Pandas using fillna

Fragments Given one pd.Series, I want to replace empty values with a list. That is, given: import numpy as np import pandas as pd ser = pd.Series([0,1,np.nan]) I want a function that can return 0 0 1 1 2 [nan] However, if I try to use the na

How to replace empty dictionary with values in pandas

Low: I have the following: pd.DataFrame({ 'a' : { 1 : {}}, 'b' : {1 : 3} }) look like: a b 1 {} 3 and would like to be able to replace it {}with 0 or NaN , but I'm not sure how to do it. I can't .replaceseem to use pd.DataFrame({ 'a' : { 1

How to replace empty dictionary with values in pandas

Low: I have the following: pd.DataFrame({ 'a' : { 1 : {}}, 'b' : {1 : 3} }) look like: a b 1 {} 3 and would like to be able to replace it {}with 0 or NaN , but I'm not sure how to do it. I can't .replaceseem to use pd.DataFrame({ 'a' : { 1

Empty dictionary in function triggers pandas error

e I have this function and it should work: def format_df(active_posts, inactive_posts, active_impressions, inactive_impressions): for name, data in vars().items(): df = pd.DataFrame(data).transpose() df.fillna(0) df[4] = df[0] / df

Append dictionary elements to an empty pandas dataframe column

data pixel I have data in pandas dataframe like this: queryName Market tags categoryDetails dummy_query (dummy_market) dummy_market dummy_tag [{'name': 'relevant_data', 'parentName': 'relevant_scrape', 'parentId': '289245228', 'id': '2892695401'},

Empty dictionary when using Pandas to_dict()

xIIPANIKIIx I am trying to create a JSON file from CSV using Pandas. I have the following function, but I'm running into an issue where the events dictionary contains nothing. import pandas as pd import json data = pd.read_csv('ufo-sightings.csv',sep = ',', d

How to replace empty dictionary with values in pandas

Low: I have the following: pd.DataFrame({ 'a' : { 1 : {}}, 'b' : {1 : 3} }) looks like: a b 1 {} 3 and would like to be able to replace it {}with 0 or NaN , but I'm not sure how to do it. I can't .replaceseem to use pd.DataFrame({ 'a' : {

Empty dictionary in function triggers pandas error

e I have this function and it should work: def format_df(active_posts, inactive_posts, active_impressions, inactive_impressions): for name, data in vars().items(): df = pd.DataFrame(data).transpose() df.fillna(0) df[4] = df[0] / df

Pandas fillna with lookup table

Zambi Having some trouble filling in NaNs. I want to have a dataframe column with several NaNs and populate them with values derived from a "lookup table" based on the values in another column. (You might recognize my data from the Titanic dataset)... Pcla

Pandas - fillna with row subset

Left__ I'm trying to fill certain rows with 0's where certain conditions apply. I'm trying now: df.loc[:,(df.Available == True) & (df.Intensity.isnull())].Intensity = df.loc[(df.Available == True) & (df.Intensity.isnull())].Intensity.fillna(0, inplace=True) T

Pandas fillna() not working

Ryan I am trying to replace NaN values in a dataframe with the mean in the same row. sample_df = pd.DataFrame({'A':[1.0,np.nan,5.0], 'B':[1.0,4.0,5.0], 'C':[1.0,1.0,4.0], 'D':[6.0,5.0,5.0],

Pandas fillna with lookup table

Zambi Having some trouble filling in NaNs. I want to have a dataframe column with several NaNs and populate them with values derived from a "lookup table" based on the values in another column. (You might recognize my data from the Titanic dataset)... Pcla

Conditional fillna() in pandas dataframe

RSM I have two dataframes below df1anddf2 df1： A B C D 1 Nora NaN Japan 2 Neo NaN India 3 Nord NaN Fuji 4 Noman 2020 Unknown df2： E F 1123 Neo 1124 Norm 1126 Nora I need to do a fillna once df1a

Is Pandas fillna wise?

Adaf Here is a simple example. d=pd.DataFrame({'x':[1,None,None,3,4],'y':[3,2,3,None,7],'z':[None,None,None,None,None]}) d['t']=d.mean(axis=1) Out[96]: x y z t 0 1.0 3.0 None 2.0 1 NaN 2.0 None 2.0 2 NaN 3.0 None 3.0 3 3.0 NaN N

Pandas fillna value increase

Eric M I have a dataframe with a column of consecutive but not adjacent numbers and missing values. I want to use the fillnafunction to fill missing values using the incremental value of the previous non-missing row. Here is a simplified table: index my_count