REST API Response with custom JSON String containing invalid characters when ran in the IBMi

[jan]

Good day,

I've encountered an issue that only happens when running REST APIs in the IBMi with the response being a custom JSON string.

Code: Select all

#uContext.Response.HttpStatus := 200
#uContext.Response.ContentType := "application/json"
#uContext.Response.ContentString := #uStringWriter.Text

The reason I went with a custom JSON String is because the array needs to be built dynamically based on data and the number of levels vary. It was a lot easier to work with it as a String rather than nested objects / arrays which kept crashing VL.

When the REST API is ran using an IIS webserver, the response does not contain the gibberish data.

So my questions would be:

1. Has anyone encountered the same issue with a custom JSON response when running REST APIs in the IBMi?
2. Is it possible to build a dynamic nested array using schema objects in the API definition?

Thanks for the help!

[Dino]

Hi Jan,

Let me try to go in parts here, as there are several questions.

Regarding crashes with the api, that is getting better with each epc, i actually applied the new epc150070 now and looks more stable. In older epcs, i got used to make changes in the api definition, then save, sometimes even exit the program, before going to work in the source.

Now... How dynamic is that nested array? Meaning, you can define nested arrays in the api definition, you dont need to use all the nested levels all the time. But you cannot create them dynamically in the api definition. (For that what you are doing which is creating a json field from scratch makes more sense).

Talking about nesting, for example, you could have an structure like department/section/employee/ and for each employee, you could have skills and/or cars. not every department need to have section with employees, neither each employee need to have skills or cars. (well, they surely need skills but that is a different subject).

If you want to define an structure like that, you can define in the schema section, different types, similar as how the template implements that, using an array and an object and playing with the reference.

Then each Array will be type array and will have items. the item in the array, will be a reference to the object.
Then each object will be type object and the properties of the object will be the fields you need in that object.
If you want to nest, a property in an object can also be a reference to another array.

nesting.png (60.75 KiB) Viewed 22128 times

[jan]

Hi @Dino,

Thank you for the response. I've actually done the sample you did in another API as I'm new in learning about the construction of Server Modules as REST APIs.

The current one with issues is actually for a list of categories and sub-categories. Sub-categories will have the same fields as a category , and the level is unknown. Sub-category levels can go as far as needed based on how the data is architectured. Within reason though, I'm guessing there would be at maximum 8 - 10 levels. Each category can have different levels of their own.

So I have an array for categories and an object.





And since the structure of a category and sub-category is the same, I initially thought that we can just reuse the same Array into the Object but Visual Lansa does not allow that (does seem sensible). But it seems to mean that I need to define a "x" number of the same Array and Object to get to a certain level in the schema.

An alternative I have not tried yet but have been thinking about is to just have the response be an Object with an Array property. Then I'll construct the nested Array data in the RDML without relying on the schema in the API definition. Haven't gotten the time to do this yet and will have to do more research on how to do it. There's also the risk that this approach may end up with the same problem as the custom JSON string.

[MarkD]

I suspect that the IBM i server “gibberish” response is correct, but it is in EBCDIC format instead of ASCII/UTF-8 format.

That’s probably because the custom setting probably turns off automatic EBCDIC -> ASCII conversion as it leaves the server.

You could check this theory by returning the test value “ABC123” and looking at the binary hex value of the result.

In ACSCII you would get 41 42 43 31 32 33

In EBCDIC it would be C1 C2 C3 F1 F2 F3

[jan]

Thanks for the response @MarkD.

I did try this with the JSON shown below. The response is too large to post here so I put it up as a .txt file.

Code: Select all

{
    "days": [
        {
            "ABC123": "ABC123"
        }
    ]
}

https://drive.google.com/file/d/1LcmktK ... sp=sharing

The response isn't readable and I'm not really sure how to translate it to something that is. I'm already over halfway through giving up on the Custom JSON String at this point and will try to work it out from the API Definitions.

[PatrickF]

Web Administrator in EPC150070 allows you to set tracing, and the Debug Option shows how the request is handled by the Apache Module

Web Admin Tracing.png (30.44 KiB) Viewed 22043 times

The screen shot below from the lansaweb.log tells me that Apache Transfer Encoding is not enabled. This means that if the JSON response is being built dynamically, Apache may not know when the content ends.

Apache Module Tracing.png (50.97 KiB) Viewed 22043 times

Here is what ChatGPT says about When Should You Use Chunked Transfer Encoding

When Should You Use Chunked Transfer Encoding.png (183.34 KiB) Viewed 22043 times

Could you enable Apache chunked transfer and then retest from PostMan ? Compare the content length in the response with the actual content