Seeking Efficient Way to Transfer Image Data from C to Xojo's Canvas

Hello everyone,

I’ve been working on a project that involves encoding Terraria world data into a PNG image using a C library and then displaying it in a Xojo Canvas. The process is functional, but I’ve encountered performance issues, especially with larger images (e.g., 8400x2400). The bottleneck seems to be the encode_png_image function, which takes a considerable amount of time.

Here’s a brief overview of the relevant parts of my code:

In C:

I have a C function encode_world_to_png that reads a Terraria world file, processes the data, and encodes it into a PNG image. The function returns a PNG image, but the encoding step is time-consuming.

// C Code

/**
 * Encodes a Terraria world file to a PNG image.
 * @param world_path The path to the Terraria world file.
 * @param img_path The path to save the PNG image.
 * @param is_mark_important Whether to mark important items in the image.
 * @return 0 if successful, 1 otherwise.
 */
int encode_world_to_png(const char *world_path, const char *img_path, bool is_mark_important) {
    FILE *fp;
    Format format;
    Header header;

    fp = topen(world_path, "rb");
    if (fp == NULL) {
        printf("Failed to open file\n");
        return 1;
    }
    parse_format(fp,&format);
    parse_header(fp, &header, &format.version,format.pointers[0],format.pointers[1]);
    // Call the read_tiles function
    uint32_t width = header.Width;
    uint32_t height = header.Height;
    uint8_t *tiles_data;
    int data_size;
    // Read tile data
    tiles_data = read_sections(fp, &format, 1, 2, &data_size);
    if (tiles_data == NULL) {
        printf("Failed to read tile data\n");
        return 1;
    }

    // Calculate the total size of RLE data
    int total_rle = width * height;
    // Calculate the size of the required image buffer
    size_t image_size = total_rle * 3; // Because RGB requires 3 bytes per pixel

    // Allocate memory for the image buffer
    unsigned char *image = malloc(image_size);
    if (image == NULL)
    {
        printf("Error allocating memory for image buffer\n");
        return 1;
    }
    set_pixels_color(image, tiles_data, &format, &header);

    if (is_mark_important) {
        uint8_t *chests_data;
        // Read chest data
        chests_data = read_sections(fp, &format, 2, 3, &data_size);
        if (chests_data == NULL) {
            printf("Failed to read chest data\n");
            return 1;
        }
        ItemColor *item_colors = NULL;
        create_item_colors_dict(&item_colors);
        mark_important_items(image, chests_data, width, height, item_colors);
        delete_item_colors_dict(&item_colors);
        free(chests_data);
    }
    fclose(fp);

    int ret = 0;
    int bit_depth = 8;
    enum spng_color_type color_type = SPNG_COLOR_TYPE_TRUECOLOR;
    printf("image_size: %d\n", image_size);
    ret = encode_png_image(img_path,image, image_size, width, height, color_type, bit_depth);
    free_format(&format);
    free_header(&header);
    free(image);
    return ret;
}

/**
 * @brief Starts an asynchronous operation to encode a Terraria world file to a PNG image.
 *
 * @param world_path The path to the Terraria world file.
 * @param img_path The path to save the PNG image.
 * @param is_mark_important Whether to mark important items in the image.
 * @param callback A pointer to the callback function to be called after the asynchronous operation completes. Can be NULL.
 * @return 0 if successful, -1 otherwise.
 */
int encode_world_to_png_async_start(const char *world_path, const char *img_path, bool is_mark_important, Callback callback);

In Xojo:

I use the MBS Xojo Plugin to call the C functions from my Xojo application. I start the asynchronous encoding process using encode_world_to_png_async_start, and a callback function is executed once the operation completes.

' In Window1
Sub Opening()
  CheckAndCreateTmpFolder
  MyLibrary =  New DeclareLibraryMBS("E:\libTerraDll.dll")
  
  Dim lines() As String = MyLibrary.SymbolNames
  MyCallback =  New Callback("i)v")
 
End Sub
' In Button1
Sub Pressed()  
  Dim file As FolderItem = OpenFileDialogSeleteTerraWorld
  If file <> Nil And file.Exists Then
    worldFileName = file.Name.TrimRight(".wld",".bak",".wld.bak")
    Dim imageFullName As CString = tmpFolderPath+"\"+worldFileName+".png"
    Dim isMarkImportant As Boolean = isMarkChestsImportant.getBoolean
    Dim worldFullName As CString = file.NativePath
    Dim p As ptr = MyLibrary.Symbol("encode_world_to_png_async_start")
    MyFounction = New DeclareFunctionMBS("(ZZBp)l", p)
    MyFounction.ParameterString(0) = worldFullName
    MyFounction.ParameterString(1) = imageFullName
    MyFounction.ParameterBoolean(2) = isMarkImportant
    MyFounction.ParameterPointer(3) = MyCallback.FunctionPtr
    MyCallback.Result = 1
    Timer1.Enabled = True
    EncodeImageProgressBar.Visible = True
    Dim r As Variant = MyFounction.Invoke
  End If
End Sub

' In Timer1
Sub Action()
  If MyCallback.Result = 0 Then
    Dim file As FolderItem
    Dim imageFullName As String = tmpFolderPath+"\"+worldFileName+".png"
    file = New FolderItem(imageFullName)
    If file <> Nil And file.Exists Then
      img = Picture.Open(file)
      imgHeight = img.Height
      imgWidth = img.Width
      scaleWidth = img.Width
      scaleHeight = img.Height
      hScrollBar.MaximumValue = imgWidth - imgCanvas.Width
      vScrollBar.MaximumValue = imgHeight - imgCanvas.Height
      imgCanvas.Refresh
      Timer1.Enabled = False
      EncodeImageProgressBar.Visible = False
    End If
  End If
  
End Sub

My Goal:

I’m looking for a more efficient approach to display Terraria world data in the Xojo Canvas. I’ve explored the possibility of directly setting pixel colors in C and passing them to Xojo without encoding to PNG. This would eliminate the time-consuming PNG encoding step.

One solution I considered was to set the pixel colors in C, bypass the PNG encoding step, and directly transfer the color data for drawing in Xojo’s canvas. I consulted GPT-4 and received [this advice :arrow_upper_right:](https://poe.com/s/zIgu9qo8VqXhpaiXvI4u). However, this solution involves setting each pixel color in C and then reading each color in Xojo to set into a Picture, which seems redundant.
Questions:

  1. Is it possible to set pixel colors in C and directly return the pixel data to Xojo for rendering in a Canvas?
  2. Are there any other suggestions or techniques for more efficiently drawing the color data processed in C onto the canvas?

I appreciate any insights or advice on how to optimize this process and improve the performance of displaying Terraria world data in the Xojo Canvas.

Thank you in advance for your help!

If you have your pixel data in a memory block, then you can probably just feed it to our TypeLib plugin’s RawBitmap.

TypeLib plugin is free. (Can find it at https://www.einhugur.com)

MemoryBlock to RawBitmap is zero cost operation since RawBitmap is just memory block with description of how the pixel data is arranged inside of it.

And RawBitmap you can then draw into your canvas, or convert it to Xojo picture using the TypeLib’s RawBitmapConverter.

If you do not have your pixel data in some sensible structure accessible in memory block then the above will obviously not work.

I have successfully reduced the processing time to within 1 second based on the methods you provided. However, this improvement has led to a new question. My images use memory of size 8400 * 2400 * 3, but the entire application occupies 315MB of space. When I haven’t drawn the image on the canvas, I suspect there might be redundant data in my program. Unfortunately, I’m unsure how to optimize it further. Could you provide me with more advice?

Below is the code for my Xojo button click event:

Dim worldPath As CString = "E:\Code\CTerra\my-c-project\wld\123.wld"
Dim isMarkImportant As Boolean = True
Dim memory As MemoryBlock = New MemoryBlock(8400 * 2400 * 3)

Dim p As ptr = MyLibrary.Symbol("encode_world_to_memoryBlock")
MyFounction = New DeclareFunctionMBS("(ZpB)l", p)

MyFounction.ParameterString(0) = worldPath
MyFounction.ParameterPointer(1) = memory
MyFounction.ParameterBoolean(2) = isMarkImportant
Dim r As Variant = MyFounction.Invoke

rawBit = New RawBitmap(memory, 8400, 2400, 0, RawBitmap.RawBitmapFormat.RGB)
Break // At this moment, the program's memory usage is 114MB, which is acceptable.
img = RawBitmapConverter.ToPicture(rawBit)
'Dim f As FolderItem = SpecialFolder.Desktop.Child("TempImage.jpg")
'img.Save(f, Picture.Formats.PNG)
imgCanvas.Refresh
 // After running up to this point, the program's memory usage surged to 315MB, which is difficult to accept.

I would appreciate any insights or suggestions you might have to help optimize the memory usage in my application. Thank you in advance for your assistance.

So, converting to the Xojo Picture costs you at least 2nd set of memory (short term) though I am not sure you can avoid that.

And that picture is then stored in memory so the Canvas knows about it…

So the biggest question maybe is if you refresh again and again and again, is the memory stable ?

Though obviously the Canvas does not need to draw picture that is 8400 x 2400 unless you have display with some awesome size !, so you might want to scale to the Canvas Size before converting to picture, which would then speed up the drawing and also reduce the memory foot print of the memory you need to store.