Skip to content

Conversation

@Andyz26
Copy link
Collaborator

@Andyz26 Andyz26 commented Apr 13, 2023

Context

Update the allocation logic in TE so that the scheduler will only put the requested node into a TE strictly using the smallest CPU cores to avoid having TEs spreading in different container sizes. Also, introduce an intermediate allocation request contract to help eventual decoupling/removal of Mesos classes.

Checklist

  • ./gradlew build compiles code correctly
  • Added new tests where applicable
  • ./gradlew test passes all tests
  • Extended README or added javadocs where applicable

@Andyz26 Andyz26 temporarily deployed to Integrate Pull Request April 13, 2023 00:40 — with GitHub Actions Inactive
@github-actions
Copy link

Test Results

126 files  ±0  126 suites  ±0   6m 15s ⏱️ +32s
535 tests ±0  526 ✔️  - 1  8 💤 ±0  1 +1 
536 runs  +1  527 ✔️ ±0  8 💤 ±0  1 +1 

For more details on these failures, see this check.

Results for commit ff1ec66. ± Comparison against base commit 247ca97.

@github-actions
Copy link

Uploaded Artifacts

To use these artifacts in your Gradle project, paste the following lines in your build.gradle.

resolutionStrategy {
    force "io.mantisrx:mantis-client:0.1.0-20230413.190432-200"
    force "io.mantisrx:mantis-common:0.1.0-20230413.190432-199"
    force "io.mantisrx:mantis-common-serde:0.1.0-20230413.190432-199"
    force "io.mantisrx:mantis-discovery-proto:0.1.0-20230413.190432-199"
    force "io.mantisrx:mantis-network:0.1.0-20230413.190432-199"
    force "io.mantisrx:mantis-remote-observable:0.1.0-20230413.190432-200"
    force "io.mantisrx:mantis-runtime:0.1.0-20230413.190432-200"
    force "io.mantisrx:mantis-runtime-loader:0.1.0-20230413.190432-200"
    force "io.mantisrx:mantis-shaded:0.1.0-20230413.190432-198"
    force "io.mantisrx:mantis-connector-iceberg:0.1.0-20230413.190432-198"
    force "io.mantisrx:mantis-connector-job:0.1.0-20230413.190432-200"
    force "io.mantisrx:mantis-connector-kafka:0.1.0-20230413.190432-200"
    force "io.mantisrx:mantis-connector-publish:0.1.0-20230413.190432-199"
    force "io.mantisrx:mantis-control-plane-client:0.1.0-20230413.190432-199"
    force "io.mantisrx:mantis-control-plane-core:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-control-plane-server:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-examples-core:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-examples-groupby-sample:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-examples-jobconnector-sample:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-examples-mantis-publish-sample:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-examples-sine-function:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-examples-synthetic-sourcejob:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-examples-twitter-sample:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-examples-wordcount:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-publish-core:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-publish-netty:0.1.0-20230413.190432-192"
    force "io.mantisrx:mantis-publish-netty-guice:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-server-agent:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-server-worker:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-server-worker-client:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-source-job-kafka:0.1.0-20230413.190432-193"
    force "io.mantisrx:mantis-source-job-publish:0.1.0-20230413.190432-193"
}

Copy link
Collaborator

@fdc-ntflx fdc-ntflx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Andyz26 Andyz26 merged commit 823474f into master Apr 14, 2023
* @param workerId worker id of the task that's going to run on the node.
* @return task executor assigned for the particular task.
*/
CompletableFuture<TaskExecutorID> getTaskExecutorFor(MachineDefinition machineDefinition, WorkerId workerId);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: What was the need to introduce a new type here?

@Andyz26 Andyz26 deleted the andyz/makeAllocationOnCoreStrict branch July 10, 2023 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants