Data buffer too small with API driver when doing JSON operation

[adaneau]

Hi all,

I am having a kafka driver for WinCC OA (inspired from the one from CERN). We want to write big JSON strings to kafka (~1MB).

But we are blocked by internal WinCC OA choice of limiting the buffer size for the driver to being a PVSSushort.

We are getting the following message :
WCCOAKafka (14), 2022.04.22 13:44:26.393, IMPL, SEVERE, 54, Unexpected state, kafkaStringTrans::toPeriph, Data buffer too small; need:814232 have:40960

Is there a reason for not using 32 bits integer for buffer? In 2022 with the amount of ram on our servers this would make more sense than limiting it to a short. Buffer definition is located in api\include\ComDrv\HWObject.hxx
Is there a way to go around this?

Thanks

BR
Alexandre

[gschijndel]

A workaround could be to store only a pointer in the buffer. In that case you will have to override the 'updateBufferLlc', 'compareLlc' and 'deleteData' functions.

[leoknipp]

I do not know a reason why the output buffer from the API to the external communication partner is limited.
Possibly the limit had no effect up to now as normally only small packages are send to the communication partner.

Best Regards
Leopold Knipp
Senior Support Specialist

[adaneau]

Hi Leo and Gertjan,

Thank you for your answers. Yes I think we are probably the first one trying to write high frequency dataset via driver.

Anyway I will give a try to Gertjan solution, except if anyone at ETM thinks that this is not a good idea

BR
Alex

[adaneau]

Ok I am now a bit lost on how to achieve this by using pointer:

I have my stringtrans class where I will save the pointer in buffer:

Code: Select all

PVSSboolean kafkaStringTrans::toPeriph(PVSSchar *buffer, PVSSuint len,
                                      const Variable &var, const PVSSuint subix) const
{
//something
  const TextVar& tv = static_cast<const TextVar &>(var);

  const TextVar* q = &tv;
  sprintf((char*)buffer, "%s", q);
}

So with this my pointer is now in buffer. This is a TextVar pointer.

But to retrieve the value I will be in my class HWService :

Code: Select all

PVSSboolean kafkaHWService::writeData(HWObject *objPtr)
{
  const TextVar* q =  objPtr->getDataPtr(); // that is not working as getDataPtr() is always returning a PVSSchar 
//something
}

So I cannot figure out how to retrieve my TextVar pointer in the other class as it is changed to PVSSchar. Do I miss something here?

BR
Alex

[gschijndel]

To assign an unrelated type pointer you will need a reinterpret cast.

Code: Select all

VariablePtr varPtr = *reinterpret_cast<Variable**>(data);

In my project I have extended my HWObject type with some functions to get and set the variable pointer.
I would derive from HmiConnection and implement the writeData function there.

[adaneau]

Hi Gertjan,

Do you use a void* pointer? Also by working in HmiConnection class, you would still have to deal with stringtrans as it is called before no? As the write data is called only after transformation and the buffer is already limited from the transformation fct toPeriph.

If you would have some example for the HWobject extension to deal with pointer that would be awesome too

Thanks a lot for help.

[gschijndel]

I do not use a void* pointer. Using the HmiConnection class will get the writeData call inside the connection instance instead of the HWService, otherwise there is not much difference in this regard. It still needs to deal with the transformation.

[kilianvp]

Is your fork public visible? I would be interested in the final solution.

[adaneau]

Hey Kilian,

No it is not. I would need to clean it a lot for this but I will consider it and keep it updated here.