[Event Hubs] How can I check the message size when using Azure IoT Hub?

[Marusyk]

Query/Question
I'm using standard IoT Hub SDKs to read messages from built-in endpoint of IoT Hub.

How can I get a full message size for analytics? I mean not only Data size (see code) but a full message size with properties.
Size that actually counted by IoT Hub.

    private async Task ProcessMessage(ProcessEventArgs eventArgs)
    {
        string equipmentNumber = eventArgs.Data.SystemProperties["iothub-connection-device-id"].ToString();
        string message = Encoding.UTF8.GetString(eventArgs.Data.Body.ToArray());
        int bodySize = message.Length;

[jsquire]

Unless I'm mistaken, the structure of your code seems to indicate use of the Event Hubs library to process messages; that signature corresponds to the EventProcessorClient handler, I believe. If I'm not correct, would you please be so kind as to share which library and which version the snippet corresponds to?

Assuming that I'm correct, there is no API within the client library to measure the size of an event in bytes. There is no public contract that specifies how either the Event Hubs or IoT Hub services store messages internally. The closest representation would be what the size of the message is when serialized to AMQP format for transport. It would be possible to measure the size of incoming messages before deserializing and propagate that, potentially.

If you're willing, I'd be interested in understanding more about your scenario and how this information would be useful to your application.

[Marusyk]

Hello @jsquire

Thank you for your response.

Unless I'm mistaken, the structure of your code seems to indicate use of the Event Hubs library to process messages; that signature corresponds to the EventProcessorClient handler, I believe.

Yes, you are right!

I'm using standard IoT Hub SDKs to read messages from built-in endpoint of IoT Hub.

Azure.Messaging.EventHubs 5.1.0
Azure.Messaging.EventHubs.Processor 5.1.0

It would be possible to measure the size of incoming messages before deserializing and propagate that, potentially.

Do you mean that this is correct, right?

string message = Encoding.UTF8.GetString(eventArgs.Data.Body.ToArray());
int bodySize = message.Length;

If you're willing, I'd be interested in understanding more about your scenario and how this information would be useful to your application.

I'm going to send the real size of each message to DataLake storage. Then, the data will be used to calculate how many Megabytes a device send per day, hour, etc. In addition, it will help us detect if the messages exceed (or almost exceed) the size of the IoT Hub messages limits

[jsquire]

Do you mean that this is correct, right?

No, I meant that the client library has access to the actual size of the message, as it was sent for transport. That includes all of the AMQP structure, overhead, and such. That also includes some properties that are owned by the broker and populated by the service when messages are requested by consumers. However, I don't believe that set of data would satisfy your requirement of understanding the IoT Hub message limits.

Just to be sure that I understand, I think what you're looking to do is understand the total size of all outgoing messages for a given device per unit of time. Did I interpret that correctly? Assuming so, can you tell me how you're sending messages from the IoT devices? My suspicion is that there's a better way to measure the metric that you're after using the IoT Hub service, but I don't know it well - so I'd like to make sure that I have good context from the sender side and then see if I can get some insight from the IoT folks.

[jsquire]

//cc: @drwill-ms - Any chance that you may have some thoughts on the best way to gather metrics on messages send to IoT Hub, in the context of a single device?