I think I've found a bug in the bulk API which causes an issue where not all fields of a record are updated.
Suppose I want to update some table in salesforce using the upsert operation. I create an array of records:
[ { property1: 'w' }, { property1: 'x', property2: 'y' }, { propety1: 'z', property2: 'a', property3: 3 } ]
Notice how different records have different properties to be updated. When I call the bulk upsert function it converts data set into a CSV. The first record of the upsert only has a single property. So the csv ends up looking something like:
property1
'w'
'x'
'z'
As a result, property 2 on the second record and property 2 and property 3 on the third record never get updated.
It looks like there's a built in assumption that all records have the same properties set on an upsert. This is definitely not true in my use cases.
A mini work around is to always have a dummy record at the beginning of your list which has all properties.