How can I force the layout to load before the data source is created?

I apologize if I ramble a bit. I am stuck between a proverbial rock and a hard place on this one.

  1. I have my app layout completed. It’s minimal and primitive, but it appears in my browser as expected…

Only If I comment out the line “data = recipe_ents_list” below.

  1. If I don’t comment it out, I receive an “Error loading layout” message. The layout will not load because it requires a data source before it will load. As a newcomer to Dash, I don’t know for sure, but it seems you can’t display a blank chart.

That is the nut of the problem:

The layout will not load without a data source, but the app does not create that source until after the layout loads.

In addition, the data source (recipe_ents_list) changes each time the app runs, meaning I cannot “prime” it.

Or can I?

Is there a way to force it to accept a null value until the user hits the submit button and generate that data?

I have pasted only the layout code below.

I will share more if requested.

(As a general overview, the user pastes a recipe into the text area above the chart. The app reads the recipe for ingredients, amounts, units of measurement, and other information pertinent to a nutritional analysis. The process is accurate, but it is not perfect. For that reason, the chart provides the user with the chance to edit/approve the information after the initial processing, producing a detailed nutritional analysis tracking 50+ nutrients.)

Thank you for reviewing my question.

app.layout = html.Div([

    dcc.Textarea(
        id='textarea-state-example',
        value='',
        style={'width': '100%', 'height': 200},
        placeholder='Please note: Information inside brackets or parantheses will be deleted.'),

    html.Button('Submit', id='textarea-state-example-button', n_clicks=0),
    html.Div(id='textarea-state-example-output',
             style={'whiteSpace': 'pre-line'}),

html.Div([
    dash_table.DataTable(
        id='table-editing-simple',
        columns=[{'id': i, 'name': i}
        for i in ['Index', 'Amounts', 'Modifiers', 'Units', 'Ings']],
        **data=recipe_ents_list,**
        editable=True
    ),
    html.Button("Submit", id="submit"),
    html.Div(id="msg"),
    dcc.Graph(id='table-editing-simple-output')
]),

])

Hi,

If I understand your problem right, you want to use the input from Textarea to generate the data that goes into the DataTable, is that so?

If so, then recipe_ents_list refers to items (ingredients) parsed from the textarea, which has to be modified via a callback. So you have to provide a callback that looks more or less like:

@app.callback(
    Output("table-editing-simple", "data"),
    Input("textarea-state-example", "value")
)
def update_table(textarea_value):
     # here you have to parse the text and return in the format
     # expected by DataTable, which is a list of dicts with keys = columns
     # and vals = values in cells
     # If you can convert to pandas, then it should be df.to_dict("records")
    return parsed_textarea_values

You can have an empty table if you pass an empty list (or even None) to the table. You probably need it if the textarea is empty or you can’t parse the content in textarea in a tabular way.

As a rule of thumb in Dash, everything that is dynamic should come from the Output of a callback.

Hope it helps!

Thank you for your time.

I spent about four hours on this problem this morning, but I think I am out of my depth.

I was already producing the library of dictionaries the table requires. The cause-and-effect was awry, though.

You had identified that part of the problem correctly.

I needed that data before the layout would load, but I could not create it until after the layout loaded. I set the data source to None, and the blank table loaded without a data source.

Because my understanding of callbacks is just not advanced enough, I created a new “Submit to Review” button and tied creating the data source recipe_ents_list to a concrete event, based on your suggested callback and code.

The new function produces and returns recipe_ents_list. If I cut and paste it into the script and call it “data,” it works. It works right up until I need Dash to recognize it as the new and valid data source.

I will not take ‘data’ as the callback Output property. And data to be the Output property (right?) because it is the variable I need to change. I keep receiving this error message:

dash.exceptions.InvalidCallbackReturnValue: The callback for `<Output `table-editing-simple.data`>`
returned a value having type `TextIOWrapper`
which is not JSON serializable.

The value in question is either the only value returned,
or is in the top level of the returned list,

and has string representation
`<_io.TextIOWrapper name='./json/recipe_ents_list.json' mode='r' encoding='cp1252'>`

In general, Dash properties can only be
dash components, strings, dictionaries, numbers, None,
or lists of those.

If I could resolve this issue, the other issues encountered this morning suggest I would still be unable to replace the placeholder data source “None” with the new data source “recipe_ents_list.”

If you have any other thoughts, I would love to hear them, of course. I am almost ready to pay someone to teach me how this works. Dash does not seem designed to support this process.

Thanks again.

I would need to see the functions you are using to convert to/from json to say what specifically is the problem, but it seems to me that you are not returning in the callback the output of json.load, which should be a dict or a list of dicts (DataTable expects the latter). I still don’t think you need this though, if I understand what you want to do correctly.

I believe it might be easier for your struggle to provide you a simple working example of the basic functionality of what you want (to my understanding). Here it is:

from dash import Dash, dcc, html, dash_table, Input, Output, callback, no_update

app = Dash(__name__)

app.layout = html.Div(
    [
        dcc.Textarea(
            id="textarea",
            value="",
            style={"backgroundColor": "white"}
        ),
        dash_table.DataTable(
            id="table",
            columns=[{"id": i, "name": i} for i in ["A", "B"]]
        )
    ]
)

@callback(
    Output("table", "data"),
    Output("textarea", "style"),
    Input("textarea", "value")
)
def update_table(value):
    rows_txt = value.split("\n")

    vals = [row.split(" ") for row in rows_txt]

    # wont update if format is wrong!
    if any([ len(row) != 2 for row in vals]):
        print("Wrong format")
        return no_update, {"backgroundColor": "salmon"}
    
    # this is the format DataTable expects
    rows = [{"A": row[0], "B": row[1]} for row in vals]

    return rows, {"backgroundColor": "white"}

    
if __name__ == "__main__":
    app.run_server(debug=True)

Here I went one step further and added a simple visual change to show that the input in textare is invalid (the background is red), but other than that is what I imagine you want to solve in your question.

You will need a different validation for the value in the textarea. I used a simple one that break lines in rows and break single spaces in “cells” delimiters… There are more robust ways to do that, but the idea remains the same.

Is this helpful?

1 Like

Wow. Thanks so much. This is so helpful. I need to sit with it, but it did work for me. I did notice you use the term json.loads. I use json.load in my code because recipe_ents_list a json object, not a string.

Is that perhaps part of the problem? It’s not been a problem in the past.

Yes. I was chastised once for sharing too much code, so I try to avoid it. Its hard to know - for me at least - where to draw that line. You are very gracious and I don’t want to impose. I need to try again before I ask more questions or share more code.

But I want to thank you for going out of your way by providing a concrete example. I learn best that way. If I succeed, I will paste the code here to help someone in the future.

Robert

1 Like

You are correct about load and not loads. I updated my anwser. You shouldn’t need it in any case, unless you really want to export to json and not just as an intermediate step.

MAJOR ANNOUNCEMENT!

Hold the presses! No need to work on this, I figured it out. I forgot to capitalize the keys in the data source. But that’s it. That was the only thing wrong. I will publish a solution soon!

I would not be the man I am today without you.

Thanks so much for everything.

I have been working on this all day. I seem very close, but I have not quite got it.

It accepts the function below, and each import produces the correct results. The callback in correctly constructed, I am almost certain. Based on error message, its trying to pass it to the table. The code produces the data, and the empty table appears without any error messages when I submit a recipe.

I know the “data” in this function, pasted below, is correctly structured, though it is not valid JSON. And that may be where the problem lies. Still, if I paste it into the script and call it “data,” it perfectly populates the table. So I am struggling to understand how it could be the problem and requires further parsing.

If I run the function as is, however, it processes the recipe and prints out the data.

But it will not populate the table.

I am at a loss and hoping you may have some final insights. It feels like I am missing something obvious, and you might spot it right away because it comes so close to working.

If this is too much, I apologize and I understand completely.

Thank you.

There is only one function and callback in the entire script now - the one shown below. Why? Because I was trying to use the output of one function and the Input of another in the next callback. And that just was not possible. Everything else is imported into this function to simplify matters and it seems to work fine until the data needs to populate the table.

1) This is the function in its current state. I could not import the list of dictionaries (recipe_ents_list) with json.load. I had to use json.loads and then then try to reconstruct it as a new list of dictionaries. If I use print(type), it tells me they are dictionaries inside a list. But it won’t accept the data unless I paste it directly into the script and call it data.


@app.callback(
    Output('nutrients-review-datatable', 'data'),
    Input('recipe-textarea-submit-button', 'n_clicks'),
    State('recipe-textarea', 'value')
)
def update_output(n_clicks, value):
    if n_clicks > 0:
        print("Recipe submitted")
        with open("temp/recipe_contents.txt", "w", encoding='utf-8') as recipe_contents:
            recipe = value.split('\n')
            for line in recipe:
                recipe_contents.write(line)
                recipe_contents.write('\n')
       #The following are my imported Python scripts and one produces the expected results.
        convert_unicode_fractions()
        convert_regular_fractions()
        prepare_recipe_text()
        parse_recipe_ner()
        prepare_ner_data_for_table()

       # Here I open recipe_ents_list and try to return it as data, this seems to be where the problem lies. I experimented with 
       # your solution for quite a while. It either does not require further parsing or I am unable to apply it.

       with open("./json/recipe_ents_list.json", "r") as final_recipe_file:
            results = json.loads(final_recipe_file.read())
            data_list = []
            for each_dict in results:
                data_list.append(each_dict)

            data = data_list

            print(data)            

            return data

2) Here are the last few lines of the final imported function, prepare_ner_data_for_table(), showing the making of recipe_ents_list - the list of dictionaries below that should become the data source. It works.

recipe_ents_list = list(unique_everseen(flask_dict_list))

        for line in recipe_ents_list:
            print("Recipe Ents List :", line)

        with open("./json/recipe_ents_list.json", "w") as final_recipe_file:
            json.dump(recipe_ents_list, final_recipe_file, indent=4)

3) This is the list of dictionaries called recipe_ents_list, as it currently stands. It is the data source that should populate the table, and this is the last printed output.

[{‘index’: 0, ‘amount’: ‘.5’, ‘mod’: ‘None’, ‘units’: ‘teaspoon’, ‘ing’: ‘dried oregano’},
{‘index’: 1, ‘amount’: ‘0.25’, ‘mod’: ‘None’, ‘units’: ‘tsp’, ‘ing’: ‘red chilli flakes’},
{‘index’: 2, ‘amount’: ‘0.25’, ‘mod’: ‘None’, ‘units’: ‘tsp’, ‘ing’: ‘ground cloves’},
{‘index’: 3, ‘amount’: ‘1’, ‘mod’: ‘None’, ‘units’: ‘tbsp’, ‘ing’: ‘sunflower oil’},
{‘index’: 4, ‘amount’: ‘1’, ‘mod’: ‘None’, ‘units’: ‘tsp’, ‘ing’: ‘mustard seeds’},
{‘index’: 5, ‘amount’: ‘1’, ‘mod’: ‘divided’, ‘units’: ‘tsp’, ‘ing’: ‘salt’},
{‘index’: 6, ‘amount’: ‘1.33’, ‘mod’: ‘None’, ‘units’: ‘tsp’, ‘ing’: ‘cumin’},
{‘index’: 7, ‘amount’: ‘1.5’, ‘mod’: ‘None’, ‘units’: ‘teaspoon’, ‘ing’: ‘dried thyme’},
{‘index’: 8, ‘amount’: ‘10’, ‘mod’: ‘None’, ‘units’: ‘teaspoon’, ‘ing’: ‘cardamom pods’},
{‘index’: 9, ‘amount’: ‘3’, ‘mod’: ‘None’, ‘units’: ‘cm’, ‘ing’: ‘ginger’},
{‘index’: 10, ‘amount’: ‘3’, ‘mod’: ‘medium’, ‘units’: ‘cm’, ‘ing’: ‘shallots’},
{‘index’: 11, ‘amount’: ‘300’, ‘mod’: ‘None’, ‘units’: ‘grams’, ‘ing’: ‘red lentils’},
{‘index’: 12, ‘amount’: ‘4’, ‘mod’: ‘minced’, ‘units’: ‘grams’, ‘ing’: ‘cloves of garlic’},
{‘index’: 13, ‘amount’: ‘400’, ‘mod’: ‘None’, ‘units’: ‘grams’, ‘ing’: ‘diced tomatoes’},
{‘index’: 14, ‘amount’: ‘80’, ‘mod’: ‘None’, ‘units’: ‘grams’, ‘ing’: ‘baby spinach’},
{‘index’: 15, ‘amount’: ‘1’, ‘mod’: ‘None’, ‘units’: ‘handful’, ‘ing’: ‘cilantro’},
{‘index’: 16, ‘amount’: ‘1’, ‘mod’: ‘Half’, ‘units’: ‘handful’, ‘ing’: ‘lemon’}]

Thank you again, I hope this is not a huge imposition.

I wish you a safe and happy holiday.

Robert

Hi @robertpfaff,

No, it is not a big imposition and I am happy to help you. I will make the most obvious suggestion first, but please let me kno if this one does not work.

I will take that you can print the data in the end of the callback and it looks like the list of dictionaries that you showed. Then if you are still defining your columns in the layout as:

columns=[{'id': i, 'name': i}
        for i in ['Index', 'Amounts', 'Modifiers', 'Units', 'Ings']]

Your problem should be that the columns index are capitalized, but your dictionary with each datum is not. Fixing the data or the columns ids should work. Note that if no keys in the datum dictionary match the ids in columns, then the datatable will be empty.

1 Like

So here is the solution to the question, how can I force the layout to load before the data source is created?

  1. As shown below, we set the data source to None. I think you could also set it to an empty list, but you can definitely set the data source to None. This allows the layout to load without an Error Loading Layout message.

data=None

  1. In the callback, we create the Input with a) the ID of the element that receives the input, like a text area in this case, and b) the action that will trigger the callback, like clicks of the submit button.

Input(‘recipe-textarea-submit-button’, ‘n_clicks’),

  1. In the callback, we create the Output with a) the ID or name of the element where we want the data to appear (e.g. a table in the case) and b) the variable we want to change, which is the data source in this example.

Output(‘nutrients-review-datatable’, ‘data’),

  1. We create a function that parses the data as desired and returns it as data (the element we want to change).

Here is the entire setup below. Both the callback and function.

@app.callback(
    Output('nutrients-review-datatable', 'data'),
    Input('recipe-textarea-submit-button', 'n_clicks'),
    State('recipe-textarea', 'value')
)
def update_output(n_clicks, value):
    if n_clicks > 0:
        print("Recipe submitted")
        with open("temp/recipe_contents.txt", "w", encoding='utf-8') as recipe_contents:
            recipe = value.split('\n')
            for line in recipe:
                recipe_contents.write(line)
                recipe_contents.write('\n')
        convert_unicode_fractions()
        convert_regular_fractions()
        prepare_recipe_text()
        parse_recipe_ner()
        prepare_ner_data_for_table()

        with open("./json/recipe_ents_list.json", "r") as final_recipe_file:
            data = json.loads(final_recipe_file.read())       

            return data

Get it?

The big takeaways are to remember that the Output is the property ID and the variable we want to change, and the function must return the data source in the format required by the table, or a list of dictionaries here.

That was precisely the problem.

It finally occurred to me that I should compare the data that was working to the data that wasn’t. I sensed it was something simple, but I was too frustrated and deep into the issue to spot the obvious.

I figured it out minutes after I wrote the last response, and quickly edited the response so that you would hopefully see the problem was solved before you worked on it. It sounds like you read the response minutes before I updated it with the solution. But it also sounds like you spotted the issue right away.

It works great now.

Thank again.