(Azure) Resolve invalid JSON Object under properties field

[d1g1tal-lore-34]

What does this PR do?

Added a function call fixPropertiesJsonString, that works on converting malformed JSON String that is part of the properties field into a proper JSON object. This issue is documented in the follow Microsoft Post.

Motivation

This will allow Datadog Log Processor to extract the proper message and log level field. This fixes issue #1014

Testing Guidelines

It has been tested in two manners:

• Tested in my Datadog Instance
• Test cases have been created under EventhubLogHandler Fix Properties Json String

Additional Notes

This issue is documented in the follow Microsoft Post.

Types of changes

• Bug fix

Check all that apply

• This PR's description is comprehensive
• This PR contains breaking changes that are documented in the description
• This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
• This PR impacts documentation, and it has been updated (or a ticket has been logged)
• This PR's changes are covered by the automated tests
• This PR collects user input/sensitive content into Datadog
• This PR passes the integration tests (ask a Datadog member to run the tests)
• This PR passes the unit tests
• This PR passes the installation tests (ask a Datadog member to run the tests)

[mattsp1290]

Overall this seems like a useful change. Thanks for putting this together. We'd just like to see a little simplification in the added function to reduce the surface of any issues.

[mattsp1290]

Suggested change

	fixPropertiesJsonString(record) {
	try {
	// Check if properties field is a string
	if (typeof record.properties === 'string') {
	// If it is a string, check if it is a malformed JSON String
	// By checking for ':'
	if (record.properties.includes("':'")) {
	// If the check is true, find / replace single quotes
	// with double quotes, to make a proper JSON
	// which is then converted into a JSON Object
	record.properties = JSON.parse(record.properties.replace(/'/g, '"'));
	return record;
	}else {
	return record;
	}
	} else {
	return record;
	}
	} catch {
	this.context.error('Unable to fix properties field to JSON Object');
	return record;
	}
	}
	fixPropertiesJsonString(record) {
	// Check if properties field is a malformed JSON String
	if (Object.hasOwn(record, 'properties') && (typeof record.properties === 'string') && (record.properties.includes("':'"))) {
	try {
	// If the check is true, find and replace single quotes
	// with double quotes, to make a proper JSON
	// which is then converted into a JSON Object
	parsedProperties = JSON.parse(record.properties.replace(/'/g, '"'));
	record.properties = parsedProperties;
	} catch {
	this.context.error('Unable to fix properties field to JSON Object');
	}
	return record;
	}

This function has a lot of return paths that all seem to do the same thing. This could be simplified a bit to have a single return and combine our checks into a single statement to verify whether or not record.properties has one of these malformed objects.

We'll need to do a little verification on our end to make this work with live data and double check the behavior of nested message and level fields.

[d1g1tal-lore-34]

@mattsp1290 I went ahead and took the changes that you've recommended. I added a couple of return clauses that ensures:

• When the if statement fails, we should return the record as is
• If the conversion fails, we should return the record as is

    fixPropertiesJsonString(record) {
        // Check if properties field is a malformed JSON String
        if (Object.hasOwn(record, 'properties') && (typeof record.properties === 'string') && (record.properties.includes("':'"))) {
            try {
                // If the check is true, find and replace single quotes
                // with double quotes, to make a proper JSON
                // which is then converted into a JSON Object
                record.properties = JSON.parse(record.properties.replace(/'/g, '"'));
                return record;
            } catch {
                this.context.error('Unable to fix properties field to JSON Object');
                return record;
            }
        } else {
            return record;
        }
    }

I have also added a couple of additional test cases, that will test nested objects.