Apply custom accumulation function to pandas


Dragon

I have the following dataframe and like to apply a custom cumulative formula to the columns.

How should I put them in a function and apply it to the new column? thank you very much!

Excel formula:

new column =(previous period value + 1) * (1 + current period of test_A's value) ^ (1/12) -1

# For example: 2021-12-11 as current period 
=(0.01 + 1 ) * (1 + 0.01 ) ^ (1/12) -1 = 0.0016598

enter image description here

Yesrel

I think it's better to use numba if performance is important:

df = pd.DataFrame({
    "test_A": [0.01] * 1000,
   
})

    
from numba import jit

@jit(nopython=True)
def f(a):
    d = np.empty(a.shape)
    p = 0
    for i in range(a.shape[0]):
        p = (1 + p) * (1 + a[i]) ** (1/12) - 1
        d[i] = p
    return d


df['Cumulative1'] = f(df['test_A'].to_numpy())
print (df)
     test_A  Cumulative  Cumulative1
0      0.01    0.000830     0.000830
1      0.01    0.001660     0.001660
2      0.01    0.002491     0.002491
3      0.01    0.003322     0.003322
4      0.01    0.004155     0.004155
..      ...         ...          ...
995    0.01    1.283884     1.283884
996    0.01    1.285778     1.285778
997    0.01    1.287675     1.287675
998    0.01    1.289572     1.289572
999    0.01    1.291472     1.291472

[1000 rows x 3 columns]

In [46]: %%timeit
    ...: df['Cumulative'] = np.nan  # create a new column
    ...: prev = 0
    ...: for i, row in df.iterrows():
    ...:     prev = (1 + prev) * (1 + row['test_A']) ** (1/12) - 1
    ...:     df.loc[i, 'Cumulative'] = prev
    ...:     
140 ms ± 998 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [47]: %%timeit 
    ...: df['Cumulative1'] = f(df['test_A'].to_numpy())
    ...: 
133 µs ± 570 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

Related


Apply custom accumulation function to pandas

Dragon I have the following dataframe and like to apply a custom cumulative formula to the columns. How should I put them in a function and apply it to the new column? thank you very much! Excel formula: new column =(previous period value + 1) * (1 + current p

Apply custom accumulation function to pandas

Dragon I have the following dataframe and like to apply a custom cumulative formula to the columns. How should I put them in a function and apply it to the new column? thank you very much! Excel formula: new column =(previous period value + 1) * (1 + current p

Apply custom accumulation function to pandas

Dragon I have the following dataframe and like to apply a custom cumulative formula to the columns. How should I put them in a function and apply it to the new column? thank you very much! Excel formula: new column =(previous period value + 1) * (1 + current p

Apply custom accumulation function to pandas

Dragon I have the following dataframe and like to apply a custom cumulative formula to the columns. How should I put them in a function and apply it to the new column? thank you very much! Excel formula: new column =(previous period value + 1) * (1 + current p

Apply custom accumulation function to pandas

Dragon I have the following dataframe and like to apply a custom cumulative formula to the columns. How should I put them in a function and apply it to the new column? thank you very much! Excel formula: new column =(previous period value + 1) * (1 + current p

Apply custom accumulation function to pandas

Dragon I have the following dataframe and like to apply a custom cumulative formula to the columns. How should I put them in a function and apply it to the new column? thank you very much! Excel formula: new column =(previous period value + 1) * (1 + current p

Apply custom function to pandas dataframe

Michael Mathews Jr. I have a pandas dataframe like this: pd.DataFrame({ 'gender': ['male', 'female', 'female', 'male', 'unknown'], 'height': [70, 61, 64, 73, 69]}) I'm trying to write a custom function that applies summary statistics to male and female height

Apply custom function to pandas df

Chique_Code I have defined the following function which searches a nested dictionary for the value of a specific key. def get_recursively(search_dict, field): fields_found = [] if len(search_dict) == 1: search_dict = search_dict[0] for k

Apply custom function to pandas dataframe

Michael Mathews Jr. I have a pandas dataframe like this: pd.DataFrame({ 'gender': ['male', 'female', 'female', 'male', 'unknown'], 'height': [70, 61, 64, 73, 69]}) I'm trying to write a custom function that applies summary statistics to male and female height

Groupby apply custom function pandas

Denman I'm trying to apply a custom function in pandas, similar to the groupby and mutate functions in dplyr. What I want to do is say given a pandas dataframe like this: df = pd.DataFrame({'category1':['a','a','a', 'b', 'b','b'], 'category2':['a', 'b', 'a',

Apply custom function to pandas df

Chique_Code I have defined the following function which searches a nested dictionary for the value of a specific key. def get_recursively(search_dict, field): fields_found = [] if len(search_dict) == 1: search_dict = search_dict[0] for k

Apply custom function to pandas dataframe

Michael Mathews Jr. I have a pandas dataframe like this: pd.DataFrame({ 'gender': ['male', 'female', 'female', 'male', 'unknown'], 'height': [70, 61, 64, 73, 69]}) I'm trying to write a custom function that applies summary statistics to male and female height

Groupby apply custom function pandas

Denman I'm trying to apply a custom function in pandas, similar to the groupby and mutate functions in dplyr. What I want to do is say given a pandas dataframe like this: df = pd.DataFrame({'category1':['a','a','a', 'b', 'b','b'], 'category2':['a', 'b', 'a',

Groupby apply custom function pandas

Denman I'm trying to apply a custom function in pandas, similar to the groupby and mutate functions in dplyr. What I want to do is say given a pandas dataframe like this: df = pd.DataFrame({'category1':['a','a','a', 'b', 'b','b'], 'category2':['a', 'b', 'a',

Apply custom function to multiple columns in pandas

Eli I'm having trouble "applying" a custom function in pandas. When I test the function, directly pass the value it works and returns the response correctly. However, when I try to pass the column value this way def feez (rides, plan): pmt4 = 200

How to apply custom scroll function to pandas groupby?

John I want to calculate daily sales from average sales using the following function: def derive_daily_sales(avg_sales_series, period, first_day_sales): """ derive the daily sales from previous_avg_sales start date to current_avg_sales end date for

How to apply custom scroll function to pandas groupby?

John I want to calculate daily sales from average sales using the following function: def derive_daily_sales(avg_sales_series, period, first_day_sales): """ derive the daily sales from previous_avg_sales start date to current_avg_sales end date for

Apply custom aggregate function to pandas DataFrame

wfgeo I have a pandas DataFrame with two float columns col_xand col_y. I want to return col_x * col_ythe sum divided bycol_x Can this be done with a custom aggregate function? I am trying to do something like this: import pandas as pd def aggregation_functio

Group and apply custom function in pandas dataframe

RGRGRG I have a DF that looks like this: I want to group by id and flag and create a new column in df with the result: [sum(value1)/sum(value2)]*12. So I need to make the result as: I created a function: `def calculation (value1, value2): result = (va

Apply custom function to multiple columns in pandas

Eli I'm having trouble "applying" a custom function in pandas. When I test the function, directly pass the value it works and returns the response correctly. However, when I try to pass the column value this way def feez (rides, plan): pmt4 = 200

Apply custom function to multiple columns in pandas

Eli I'm having trouble "applying" a custom function in pandas. When I test the function, directly pass the value it works and returns the response correctly. However, when I try to pass the column value this way def feez (rides, plan): pmt4 = 200

How to apply custom scroll function to pandas groupby?

John I want to calculate daily sales from average sales using the following function: def derive_daily_sales(avg_sales_series, period, first_day_sales): """ derive the daily sales from previous_avg_sales start date to current_avg_sales end date for