Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"error during cell creation" when adding item in data_editor with dynamic row add. #7458

Closed
3 of 4 tasks
WilsonCampbell-BMOC opened this issue Sep 29, 2023 · 4 comments · Fixed by #8640
Closed
3 of 4 tasks
Assignees
Labels
feature:st.data_editor priority:P2 status:confirmed Bug has been confirmed by the Streamlit team type:bug Something isn't working

Comments

@WilsonCampbell-BMOC
Copy link

Checklist

  • I have searched the existing issues for similar issues.
  • I added a very descriptive title to this issue.
  • I have provided sufficient information below to help reproduce this issue.

Summary

I’ve created a data_editor that displays data pulled from a dynamo db database. I’ve created a function to capture when a row is added in order to add this new item to the database. This function handles edits and deletions also which work really well with making live changes and deletions to the dynamo db with boto3 library.

The function for handling the changes to the dataframe via the data_editor fires in the “on_change” callback. I started testing figuring out the key creation before sending it to the database. During this testing process, I clicked on the add row button at the bottom of the data_editor and this error popped up:

9c5a975b41115ca975ecdc7b90f3990dd3c21cd5_2_689x244

The addition of the row was captured in the dictionary that captures those, but I can’t do anything else from here.

Reproducible Code Example

Here is the function data_editor function that produces the data_editor

def display_table_data():
    """
    For displaying the data in the current active table.
    This uses Streamlit's data_editor() object.
    # TODO - add "save edits" button to become active when changes are made in the data_editor
    :return:
    """
    # - store the active table object for ease of use in function
    active_table = st.session_state['active_table']

    # - now get the current page, which will be the "active DataFrame"
    if 'current_page' in st.session_state:
        current_page = st.session_state['current_page']
        # - store the active DataFrame for the active table
        st.session_state['active_dataframe'] = st.session_state['active_table_data_list'][active_table][current_page]

    else:

        current_page = ''

    # - generate the data_editor table to display the data
    # -     store is in the 'edited_df' where any changes are stored as a full df
    # -     the individual edits made will be stored in st.session_state['work_dict']
    st.session_state['edited_df'] = st.data_editor(
        data=st.session_state['active_dataframe'],
        key='work_dict',
        hide_index=False,
        num_rows='dynamic',
        on_change=live_edits,
        use_container_width=True
    )

    if current_page != '':
        # create columns
        col1, col2 = st.columns([4, 1])

        with col1:

            if current_page != 0:

                st.button(label="Previous Page:", key="previous_page_button")

        with col2:

            if current_page != len(st.session_state['active_table_data_list'][active_table])-1:

                st.button(label="Next Page:", key="next_page_button")

            else:

                if "last_key" in st.session_state:
                    st.button(label="Load Next Page:", key="load_next_page_button")


here is the page where the data_editor is called:

import streamlit as st

import openpyxl
import pandas as pd
import os
import datetime as dt
import re
import zipfile
from tools import client_management_tools as cmt
from tools import dbmanager as db

# ---- START Page Settings and resources
st.set_page_config(
    layout="wide",
    initial_sidebar_state="collapsed"
)

# - Establish database connection.
db.connect()

# - get table for this page.
if 'active_table' not in st.session_state:
    db.get_active_table('client_management')

# - Now get the table data, so it is available to the script
if 'active_table_data_list' not in st.session_state:
    db.get_table_data(paginate=True)

# ---- END Page Setting and resources

# ---- START User Interface

# - display new client form
with st.expander("Add New Client"):
    with st.form(key="new_client_form"):

        st.subheader("Add New Client")

        st.text_input(
            label="Enter Full Client Name:",
            key="full_client_name",
            placeholder="Enter Name with First Letter Capitalized"
        )

        st.date_input(
            label="Enter Date of Onboarding:",
            key="onboarding_date",
            help="Select the date the client started working with BMOC."
        )

        st.text_input(
            label="Enter Client Abbreviation or Short name:",
            key="client_abbr",
            placeholder="Enter as all caps, example: TUFTS, AU, BMOC"
        )

        new_client_added = st.form_submit_button(
            label="Add Client"
        )

if new_client_added:

    new_data = {'fields': {
        'client_name': st.session_state['full_client_name'],
        'client_add_date': dt.datetime.strftime(st.session_state['onboarding_date'],
                                                '%B %d, %Y'),
        'client_abbreviation': st.session_state['client_abbr']
    }}

    db.add_item(add_key=True, item_data=new_data)

# - Display the data editor
db.display_table_data()

zip_file = cmt.show_package_export()

cmt.show_file_download()


# ---- END User Interface

# ---- START Activities of the page




db.get_next_page()
db.get_previous_page()


# ---- END Activities of the page

Here is the function for capturing the changes and sending them to the dynamodb in its draft testing form:

def live_edits():
    """
    Function for capturing edits made using the streamlit data_editor widget
    This function is set to fire when the dataframe, is edited in the data_editor widget itself.

    idea is to get the edited/changed data from the data_editor, and then use the standard
    add, edit, delete functions for dynamo db to make the changes.

    not necessarily anything passed to this function as parameters due to capturing of changes
    in the working dictionary when the data_editor widget of streamlit is in use.

    Will need to use this to record changes. Have to figure out a separate change function
    to capture changes made and store in a change_tracker table.

    Returns
    -------

    """
    active_table = st.session_state['active_table']

    changes = st.session_state['work_dict']

    if len(changes['deleted_rows']) > 0:

        for deleted in changes['deleted_rows']:

            if 'active_table_data_list' in st.session_state:
                # - set the current page back to the first of the paginated pages
                current_page = st.session_state['current_page']
                # - now to replace the existing DF in the 0 current page spot.
                adj_df = st.session_state['active_table_data_list'][active_table][current_page]

            elif 'active_dataframe' in st.session_state:
                # - else, check for "active_dataframe" in st.session_state which is data_editors
                adj_df = st.session_state['active_dataframe']
            else:
                pass

            # - capture the item from its original dataframe to get keys.
            item = adj_df.reset_index().iloc[deleted].to_dict()

            # - Get the keys to pass to the delete function
            if 'partition_key' in st.session_state['active_table_keys']:

                partition_name = st.session_state['active_table_keys']['partition_key']
                # - capture the partition key value from the DF
                partition_value = item[partition_name]

            else:

                st.error('A partition Key and its value could not be captured.')
                return


            # - get the sort key and val if there is one
            if 'sort_key' in st.session_state['active_table_keys']:

                sort_key_name = st.session_state['active_table_keys']['sort_key']
                # - capture the partition key value from the DF
                sort_key_value = item[sort_key_name]

            else:

                sort_key_name = None
                sort_key_value = None


            delete_item(
                partition_key=partition_name,
                partition_val=partition_value,
                sort_key=sort_key_name,
                sort_key_val=sort_key_value
            )

    if len(changes['edited_rows']) > 0:

        for index, changes_made in changes['edited_rows'].items():

            if 'active_table_data_list' in st.session_state:
                # - set the current page back to the first of the paginated pages
                current_page = st.session_state['current_page']
                # - now to replace the existing DF in the 0 current page spot.
                adj_df = st.session_state['active_table_data_list'][active_table][current_page]

            elif 'active_dataframe' in st.session_state:
                # - else, check for "active_dataframe" in st.session_state which is data_editors
                adj_df = st.session_state['active_dataframe']
            else:
                pass

            # - capture the item from its original dataframe to get keys.
            item = adj_df.reset_index().iloc[index].to_dict()

            # - Get the keys to pass to the delete function
            if 'partition_key' in st.session_state['active_table_keys']:

                partition_name = st.session_state['active_table_keys']['partition_key']
                # - capture the partition key value from the DF
                partition_value = item[partition_name]

            else:

                st.error('A partition Key and its value could not be captured.')
                return

            # - get the sort key and val if there is one
            if 'sort_key' in st.session_state['active_table_keys']:

                sort_key_name = st.session_state['active_table_keys']['sort_key']
                # - capture the partition key value from the DF
                sort_key_value = item[sort_key_name]

            else:

                sort_key_name = None
                sort_key_value = None

            update_item(
                partition_key=partition_name,
                partition_val=partition_value,
                sort_key=sort_key_name,
                sort_key_val=sort_key_value,
                changed_data=changes_made
            )

    if len(changes['added_rows']) > 0:

        st.write(changes['added_rows'])
        for index, added in enumerate(changes['added_rows']):
            st.dataframe(st.session_state['edited_df'])
            new_row = changes['added_rows'][index]

            if len(new_row) > 0:
                # - generate a new key to plug in to the new item
                record_key = generate_random_key()

                # - Get the name of the partition key for the active table
                if 'partition_key' in st.session_state['active_table_keys']:

                    partition_name = st.session_state['active_table_keys']['partition_key']

                else:

                    st.error('A partition Key and its value could not be captured.')
                    return

                # - Get the name of the sort key if there is one for the current table
                if 'sort_key' in st.session_state['active_table_keys']:

                    sort_key_name = st.session_state['active_table_keys']['sort_key']

                else:
                    sort_key_name = None

                # - if there is no sort key, then the partition key is a random key
                if sort_key_name is None:
                    # - if no sort key, then the partition key gets the rando
                    new_row[partition_name] = record_key

                else:
                    # - if there is a sort key, then it gets the rando
                    new_row[sort_key_name] = record_key

                # - now capture the first
                st.session_state['active_dataframe']


                #
                # item_find = tinydb.Query()
                # the_table.upsert(
                #     new_row,
                #     item_find['key'] == record_key
                # )

Steps To Reproduce

Just simply clicked on the add row part of the data_editor and the error popped up. That should be all the is required to get it to occur again.

Expected Behavior

What I expect to happen at this point is just to start putting in entries into this new row.

Current Behavior

It simply displays in each cell “Error during cell creation” and when I hover over it, it shows a pop up that says: “This should never happen. Please report this bug. Error: Error: Row index is out of range: 7”

i noted that there is no index provided in the index column of the new row.

Is this a regression?

  • Yes, this used to work in a previous version.

Debug info

Streamlit version: 1.27.0
Python version: Python 3.11
Using PyCharm environment with pycharm version 2023.2.1 (professional edition)
OS version: macOS Catalina version 10.15.7
Browser version: Chrome, Version 117.0.5938.92 (official building) (x86_64)

Additional Information

pandas>=1.5.1
openpyxl>=3.1.0
tinydb>=4.7.0
datetime>=4.0
streamlit>=1.21.0
pillow>=9.5.0
XlsxWriter>=3.0.8
boto3>=1.24.5

requests>=2.31.0

@WilsonCampbell-BMOC WilsonCampbell-BMOC added status:needs-triage Has not been triaged by the Streamlit team type:bug Something isn't working labels Sep 29, 2023
@github-actions
Copy link

If this issue affects you, please react with a 👍 (thumbs up emoji) to the initial post.

Your feedback helps us prioritize which bugs to investigate and address first.

Visits

@kossimon-mama
Copy link

This seems to be connected to use of st.expander as a container for the editor - I have the same issue / I use expander / and another issue
https://discuss.streamlit.io/t/streamlit-st-data-editor-error-in-cell-creation/46868/4

also uses expander to house the editor.

Using expander makes sense for such application with editing data frames...

@zoltan-spire
Copy link

I am experiencing the same error in a non-deterministic fashion (mostly it's OK, but sometimes the error pops up without any apparent reason), and I am not using expander anywhere in my application.

@LukasMasuch LukasMasuch added status:confirmed Bug has been confirmed by the Streamlit team priority:P2 and removed status:needs-triage Has not been triaged by the Streamlit team labels May 10, 2024
@LukasMasuch
Copy link
Collaborator

LukasMasuch commented May 10, 2024

Thanks for reporting this issue, and sorry for the delay. I reproduced this here and started a fix here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature:st.data_editor priority:P2 status:confirmed Bug has been confirmed by the Streamlit team type:bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants