Skip to content

Conversation

nji302
Copy link

@nji302 nji302 commented Dec 19, 2023

Flink within Lyft doesn't actually use the master branch, and future changes for Flink 1.17 can be made directly on the release-1.17-lyft branch. Mainly creating this PR so the team sees the Flink 1.17 changes and has visibility.

vancior98 and others added 30 commits March 3, 2023 20:07
…gg-function.enabled'

This closes apache#21789

(cherry picked from commit afdc079)
…alue when the argument type is string

This closes apache#22031

(cherry picked from commit 263555c)
…native-agg-function.enabled option can't turned on per job when using it via SqlClient

This closes apache#22031

(cherry picked from commit 62a3b99)
- Only obtain execution exception if the job is in globally terminal state

- [hotfix] Unsubscribe finished TestEventSources from test commands.  Otherwise,
  any command with SINGLE_SUBTASK scope might be dispatched to a finished source.
  This will result in TestJobExecutor.waitForFailover timing out while waiting for the
  command to be executed and ACKed.

- [hotfix] Mark TestEventSource.scheduledCommands volatile

- [hotfix] Make sure to process all commands in TestEventSource
…ger constructor and mark it as deprecated

This closes apache#22094.
…sk manager in FineGrainedSlotManagerTest.

We use TaskManagerTracker.addPendingTaskManager to allocate task manager
in FineGrainedSlotManagerTest. This make the resource requirements
different between FineGrainedSlotManager and TaskManagerTracker. And the
FineGrainedSlotManager may trigger resource recycling after the requirementCheckDelay.

So we use processResourceRequirements to allocate task manager. And set
the requirementCheckDelay to 0 to stabilize this test.

(cherry picked from commit bec6a45)
…ns in column list of a partial-insert statement

This closes apache#22109
…leUp timing out.

We're starting TMs with two slots, and in case both slots don't arrive simultaneously, we might have an additional restart of the job. Unfortunately calling OnceBlockingNoOpInvokable#cancel would effectively unblock all future #invoke calls by setting the running flag to true. We address the issue by tracking this flag for each execution attempt separately.
… causes issues with Hugo version 0.111.0 and higher. This closes apache#22113

(cherry picked from commit 3ea83ba)
…eadSchedulerTest#testRequestTimeoutIsRefreshedAndSuccess.

(cherry picked from commit a306831)
…f numRequestedBuffers is greater than 0.

This will cause deadlock as read buffer was occupied by a reader with low consumption priority.

This closes apache#22122

(cherry picked from commit 5ad2ae2)
…tLocalHostThrows swallows IllegalArgumentException
…rtitionGroup if it has too many consumers

This closes apache#22098.
…huffle

Currently, the SortMergeResultPartition may allocate more network buffers than the guaranteed size of the LocalBufferPool. As a result, some result partitions may need to wait other result partitions to release the over-allocated network buffers to continue. However, the result partitions which have allocated more than guaranteed buffers relies on the processing of input data to trigger data spilling and buffer recycling. The input data further relies on batch reading buffers used by the SortMergeResultPartitionReadScheduler which may already taken by those blocked result partitions that are waiting for buffers. Then deadlock occurs. This patch fixes the deadlock issue by reserving the guaranteed buffers on initializing.

This closes apache#22148.
…" from table header of operator IDs

This closes apache#22164.

Co-authored-by: jingge <[email protected]>
…wnload links on the the FileSystem documentation. This closes apache#22039

Remove Scala suffix for ORC and Parquet format download links on the the FileSystem documentation

(cherry picked from commit a8848f8)
…artitionReadSchedulerTest.testRequestBufferTimeout

This closes apache#22172.
…t#testShutDown.

testShutDown using two gateways contains a timeout gateway but expected all request futures not done before shutdown. If the timeout reached, this future will be failed, result in request future to be done.
…ngly pushed down and get wrong join results

This closes apache#22049

(cherry picked from commit 8990822)
nji302 and others added 30 commits January 30, 2024 06:14
This reverts commit a42c5c5.
[FLINK-32414][connectors/common] Don't emitLatestWatermark when lastEmittedWatermark is UNINITIALIZED
[hotfix] Migrate SourceOperatorAlignmentTest to AssertJ
Flink parallel checkpoint deletion cherrypick
Revert "Flink parallel checkpoint deletion cherrypick"
…herrypick

Cherrypick of parallel checkpoint deletion change
…deletion_cherrypick

Revert "Cherrypick of parallel checkpoint deletion change"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet