Writing an image taking too long

Dash Python
1

I am creating an app that takes in information, makes multiple graphs, and then prints them on a pdf. But when I go to write an image it doesn’t seem to respond. I have logging incorporated to track and it always stops at the write_image() part.

I know the graph is working because when I’ve put fig.show() before the saving part, it graphs it. It also gives no error.

Here is my function:

def make_graphs_bulk(region: str, report: str):
    # Cycle through engine names in engine_names_regions table to retrieve all names for a specific region
    logging.info('Getting engine list')
    engine_df = sql.get_engine_names_by_region(region)
    engine_name_list = [name for name in engine_df['name']]

    # Check POD and get saved data for that report.
    logging.info('Getting POD info')
    pod_df = pod.get_pod_info(report)
    print(pod_df)

    outdir = f'./reports/images/{region}'
    if not os.path.exists(outdir):
        os.mkdir(outdir)

    i = 0
    while i < len(engine_name_list):
        # If there are adjustments, call to sqlite to get up to date data and graph adding in the adjustments and save graph
        if engine_name_list[i] in pod_df['EngineName'].values:
            engine_name = engine_name_list[i]
            logging.info(f'Getting info for {engine_name}')
            engine_row = pod_df[pod_df['EngineName'] == engine_name]
            start_date = pd.to_datetime(engine_row['StartDate'].values[0], infer_datetime_format=True)
            end_date = pd.to_datetime(engine_row['EndDate'].values[0], infer_datetime_format=True)
            if pod_df['LowerBound'].values[0] is not None:
                lower = float(pod_df['LowerBound'])
            else:
                lower = -15
            if pod_df['UpperBound'].values[0] is not None:
                upper = float(pod_df['UpperBound'])
            else:
                upper = 15
            
            offset = pod_df['Offset'].values[0]

            # Get data for the engine
            logging.info('Retrieving data')
            data_df = sql.get_data_by_engine(engine_name, region)

            # Convert date column to datetime
            data_df['date'] = pd.to_datetime(data_df['date'])

            # Cut dataframe to only include dates in the time frame given from POD
            logging.info('Cutting dataframe')
            mask = (data_df['date'] >= start_date) & (data_df['date'] <= end_date)
            data_df = data_df.loc[mask]
            data_df.reset_index(inplace=True)

            # Generate graph
            logging.info('Graphing')
            fig = graph(df=data_df, engine_name=engine_name, report=report, lower=lower, upper=upper, offset=offset)
            # fig.show()

            # Save graph to file
            outname = f'{engine_name}.png'
            fullname = os.path.join(outdir, outname)
            fig.write_image(fullname)
            logging.info(f'Graph of {engine_name} saved')

            i += 1

        else:
        # If there are not adjustments, create an image that says not analyzed
            engine_name = engine_name_list[i]
            logging.info(f'Graph of {engine_name} not created')

            img = Image.new('RGB', (500, 300), (255, 255, 255))
            draw = ImageDraw.Draw(img)
            font = ImageFont.truetype("arial.ttf", 50)
            text_width, text_height = draw.textsize(f"{engine_name} \nNot Analyzed", font=font)
            x = (img.width - text_width) / 2
            y = (img.height - text_height) / 2
            draw.text((x,y), f"{engine_name} \nNot Analyzed", (0, 0, 0), font=font)
            
            outname = f'{engine_name}.png'
            fullname = os.path.join(f'../reports/imgs/{region}', outname)
            img.save(fullname)

Any thoughts why it’s not writing?

2

For the record, it works if I change it to an html and use write_html().

Ughhh.

3

So you need a static file? *.png?

4

Yes I do. I’m not sure if I’m doing something wrong or if it’s the write image function. It’s worked for me before with this exact graph from an old project and it didn’t take long at all which is why I’m confused.

5

You could try to_image()

 to_image(*args, **kwargs)

    Convert a figure to a static image bytes string

https://plotly.com/python-api-reference/generated/plotly.graph_objects.Figure.html

6

Looking at it right now. The write_image takes the argument of a filepath to save it. How do you implement that here?

7

That’s how i did this in the past, there might be other (better) ways to do it, though.

8

Gotcha. Well I was debugging and to_image() still isn’t working either :confused:

9

I’m looking at Kaleido and it seems to be a common issue with no fix? Does anyone know a work around?

10

Looks like I had to download psutil and downgrade to kaleido==0.1.0.post1

Oof.

1 Like
11

Kaleido has some major hanging issues, this is exaggerated on low memory systems.
This is likely what you were experiencing.

See here:

1 Like
12

I had a similar issue in windows 11. Had to downgrade to kaleido==0.1.0.post1. However, if you’re trying to convert big data to static image (in my case to avoid slow interactive charts), there is a hard limit. In one of my plots, a scatter overlayed with density distribution scatter plus marginal violins, the input table can’t have > ~1.8e6 rows. Apparently, that is fixed in latest kaleido version, but latest kaleido hangs in W11 during image conversion (last time I checked).