Fix worker metadata format

[liliankasem]

We'd like to be more strict about format for the analytics team.

We currently see this in the metrics table:

"RuntimeName": node,
"RuntimeVersion": v16.16.0,
"WorkerVersion": 3.5.0,
"WorkerBitness": ia32

Is it possible to remove the version prefix ("v") from runtimeVersion?
We also noticed the bitness looks wrong: "ia32" which should be "x86"

[ejizba]

We also noticed the bitness looks wrong: "ia32" which should be "x32"

It's debatable what is "wrong" or right. Here's one stackoverflow post discussing some differences. Assuming you're used to windows terminology, I think it's saying ia32 is the same as x86, but not x32. In any case, here is the list of possible arch values given to us by Node.js and I'd rather not manipulate those if we don't have to

Is it possible to remove the version prefix ("v") from runtimeVersion?

Similar to above, we're just passing along the version given to us by Node.js so I'd prefer not to modify it. Sure we could remove the 'v', but couldn't the analytics team do that as well? It just feels risky if the analytics team is going to rely on these properties being in a strict format across all workers

[liliankasem]

Thanks for the info, that's totally fair and makes sense to me. I think the goal was to make this easier for the analytics team for the adoption dashboard this data is going to be used for.

cc: @fabiocav @cloudmelon @jcookems FYI

[jcookems]

@ejizba, I think you summed it up well: should each layer pass along what the previous one gives us, or should some layer "fix" the value to make them consistent? (The Analytics will follow your example and "just pass along the version given to us" and have the next layer handle it, the dashboard in this case)

@liliankasem, I think the only reason we'd want enforce consistency would be if the Functions team wants to look at things like workerBitness across runtimes in the resulting dashboards. We might need to get Melony's input on this.
It doesn't make sense to compare versions across runtimes. In general, I'd expect that each runtime's records would only be compared with other records for the same runtime, so cross-runtime diffs probably wouldn't by problematic.

If consistency is needed, it should happen at the lowest level, where the details like Node.js's "list of possible arch values" are known, instead of relying on a third-party (Analytics) to grok those details for all Functions runtimes.

[ejizba]

@lilyjma will sync with the PM team on this as it seems like a problem for all languages

[fabiocav]

@lilyjma let's follow up on this issue. We're trying to have items associated with this feature closed out.

[lilyjma]

Replied to existing email thread about this. I think this is a cross language question that needs the other language's input as well.

[liliankasem]

@cloudmelon any updates on how this issue is going? Has there been an agreement on format across language workers?

[cloudmelon]

Yes. I spoke with Lily and she agreed to align with us and will follow up with Eric, Kind regards Melony

…

________________________________ From: Lilian Kasem ***@***.***> Sent: Wednesday, March 15, 2023 1:55:59 PM To: Azure/azure-functions-nodejs-worker ***@***.***> Cc: Melony Qin ***@***.***>; Mention ***@***.***> Subject: Re: [Azure/azure-functions-nodejs-worker] Fix worker metadata format (Issue #633) @cloudmelon<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcloudmelon&data=05%7C01%7CMelony.Qin%40microsoft.com%7C02a0d9efcb83443f977008db2597ae31%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638145105632545064%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qsnvSkiqB8vear4YPNxujZCynmzoVoxU%2FRCUG9dUWpM%3D&reserved=0> any updates on how this issue is going? Has there been an agreement on format across language workers? — Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-functions-nodejs-worker%2Fissues%2F633%23issuecomment-1470828805&data=05%7C01%7CMelony.Qin%40microsoft.com%7C02a0d9efcb83443f977008db2597ae31%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638145105632545064%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=uiIEHneggfs7LbhIlS9jRV7I2J5DYIbLv3GgVDsLEh0%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABDIJ6HL6VRWOR4PQ4O5P33W4IUF7ANCNFSM6AAAAAAQ47GXZQ&data=05%7C01%7CMelony.Qin%40microsoft.com%7C02a0d9efcb83443f977008db2597ae31%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638145105632701745%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vwpLw3GABSWA9YAnSMuJiywA53PtOldxVWJPgfa8pgU%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ejizba]

I updated the version to remove the 'v' here: #658. That one was easy enough so I just did it

The bitness is more complicated with nuances between .NET and Node.js, so we haven't had a compelling enough reason to work on that. After following up with me, @lilyjma started an offline thread with @fabiocav and @cloudmelon a few months ago but it did not receive a response.

Perhaps if you explained how this data is meant to be used outside of the worker teams that would be more compelling? My general view is that this information is most helpful for the language worker crew (tracking their own telemetry, investigating incidents, etc.), in which case it would be best (and easiest) to keep it in the language’s “native” format.

[cloudmelon]

I updated the version to remove the 'v' here: #658. That one was easy enough so I just did it

The bitness is more complicated with nuances between .NET and Node.js, so we haven't had a compelling enough reason to work on that. After following up with me, @lilyjma started an offline thread with @fabiocav and @cloudmelon a few months ago but it did not receive a response.

Perhaps if you explained how this data is meant to be used outside of the worker teams that would be more compelling? My general view is that this information is most helpful for the language worker crew (tracking their own telemetry, investigating incidents, etc.), in which case it would be best (and easiest) to keep it in the language’s “native” format.

hi @ejizba i had a very long convo with @lilyjma about this, and I thought she talked with you already. Hi @lilyjma were you able to follow up on this ( after our convo a while back ) ? thanks!

[liliankasem]

I updated the version to remove the 'v' here: #658. That one was easy enough so I just did it
The bitness is more complicated with nuances between .NET and Node.js, so we haven't had a compelling enough reason to work on that. After following up with me, @lilyjma started an offline thread with @fabiocav and @cloudmelon a few months ago but it did not receive a response.
Perhaps if you explained how this data is meant to be used outside of the worker teams that would be more compelling? My general view is that this information is most helpful for the language worker crew (tracking their own telemetry, investigating incidents, etc.), in which case it would be best (and easiest) to keep it in the language’s “native” format.

hi @ejizba i had a very long convo with @lilyjma about this, and I thought she talked with you already. Hi @lilyjma were you able to follow up on this ( after our convo a while back ) ? thanks!

I'll let Lily follow up with more details here but the tl;dr is that the analytics team is using this data to create an adoption dashboard; last I heard is that the data is not reliable enough to start ingesting. We're working to make sure all the workers follow the same data format/contract to unblock this.