DDP-5479 Stream GBF Updates #80
Conversation
…hrough them, stream through the data, updating as we go. Along the way, to fix some non-transactional behavior, it was necessary to add `Connection` param to many methods, which rippled into a number of classes as this is a fairly wide refactor in addition to the streaming change.
| ArrayList<KitRequest> kitRequests = shipper.getKitRequestsNotDone(kitType.getInstanceId()); | ||
| shipper.orderStatus(kitRequests); | ||
| kitRequests.clear(); | ||
| shipper.updateOrderStatusForPendingKitRequests(kitType.getInstanceId()); |
There was a problem hiding this comment.
Instead of creating a large object here and passing it around, this changes things to streaming.
| else { | ||
| logger.error("delivered kitDDPNotification was null for " + kit.getExternalOrderNumber()); | ||
|
|
||
| if (!isReturn) { |
There was a problem hiding this comment.
Not much is really changed here--just passing in a Connection and adjusting accordingly.
| stmt.setLong(2, externalOrderDate); | ||
| stmt.setString(3, dsmKitRequestId); | ||
| stmt.setString(4, externalOrderStatus); | ||
| public static void updateKitRequest(Connection conn, String externalOrderStatus, long externalOrderDate, String dsmKitRequestId) { |
There was a problem hiding this comment.
Another pass-in-the-Connection change is the only thing going on here.
|
|
||
| public void orderCancellation(ArrayList<KitRequest> kitRequests) throws Exception; | ||
|
|
||
| public ArrayList<KitRequest> getKitRequestsNotDone(int instanceId); |
There was a problem hiding this comment.
I moved this to GBFRequestUtil, rather than the interface, since it's now used only when testing GBFRequestUtil.
| for (String dsmKitRequestId : dsmKitRequestIds) { | ||
| KitRequestExternal.updateKitRequest(status.getOrderStatus(), System.currentTimeMillis(), dsmKitRequestId);// in order to update time for the next 24 hour check we need this | ||
| } | ||
| public void orderStatus(Connection conn, KitRequest kit) throws Exception { |
There was a problem hiding this comment.
Rather than iterating on a list, this now operates on a single kit at a time, using the passed-in Connection.
| rs.getLong(DBConstants.EXTERNAL_ORDER_DATE)); | ||
| numOrdersProcessed.incrementAndGet(); | ||
| try { | ||
| orderStatus(conn, kitRequest); |
There was a problem hiding this comment.
Wrapping this in a try/catch so we can keep going even if we run into a problem for a single kitRequest.
| // uploadKit(); | ||
|
|
||
| ExternalShipper shipper = (ExternalShipper) Class.forName(DSMServer.getClassName("gbf")).newInstance();//GBFRequestUtil | ||
| GBFRequestUtil shipper = new GBFRequestUtil(); |
There was a problem hiding this comment.
No need to instantiate via reflection, since we know exactly the GBFRequestUtil class we're testing here.
| String sendRequest = DSMServer.getBaseUrl(getExternalShipperName()) + STATUS_ENDPOINT; | ||
| logger.info("payload: " + payload.toString()); | ||
| Response gbfResponse = executePost(Response.class, sendRequest, payload.toString(), DSMServer.getApiKey(getExternalShipperName())); | ||
| logger.info(gbfResponse.getXML()); |
There was a problem hiding this comment.
This might be blowing things up--big log of a potentially very large string of XML.
3 participants
For DDP-5479, Instead of extracting long lists of
KitRequestsand passing them around--which is contributing to memory pressure--these changes stream things a bit more so that as we work through the result set of pending kits, we update its status from GBF as we go.While making these changes, I found another latent bug that can result in inconsistent transactions. I fixed this by passing around
Connections, which rippled through a number of classes unrelated to the driving changes above.I also suspect that some of our larger log statements--which pull fairly large objects into a
String--may be contributing to our problems, so I removed those.