Skip to content

ScrollAll() #2443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Dec 7, 2016
Merged

ScrollAll() #2443

merged 2 commits into from
Dec 7, 2016

Conversation

@Mpdreamz
Copy link
Member

@Mpdreamz Mpdreamz commented Dec 2, 2016

Exposes a sliced scroll behind an observable with an optional max degree
of parallelism

var scrollObservable = this._client.ScrollAll<SmallObject>("1m", slices, s => s
	.MaxDegreeOfParallelism(slices/ 2)
	.Search(search => search
		.Index(index)
		.AllTypes()
		.MatchAll()
	)
);

russcam
russcam suggested changes Dec 7, 2016
View reviewed changes
Copy link
Contributor

@russcam russcam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good, just a few small nitpicks 😄

Are there tests for the ScrollAll observable too?

src/Nest/Document/Multiple/Reindex/ReindexResponse.cs
public ISearchResponse<T> SearchResponse { get; internal set; }

public int Scroll { get; internal set; }
public long Scroll { get; internal set; }
Copy link
Contributor

@russcam russcam Dec 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch 👍

src/Nest/Document/Multiple/ScrollAll/ScrollAllObservable.cs
private readonly ISearchRequest _searchRequest;
private readonly IConnectionSettingsValues _connectionSettings;
private readonly IElasticClient _client;
private static IList<ISort> DocOrderSort = new ReadOnlyCollection<ISort>(new List<ISort> { new SortField { Field = "_doc" } });
Copy link
Contributor

@russcam russcam Dec 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should DocOrderSort be applied only if a sort is not specified on the search request? If it will always be applied, I think it should also be mentioned in the comment on Search on IScrollAllRequest

Copy link
Member Author

@Mpdreamz Mpdreamz Dec 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

src/Nest/Document/Multiple/ScrollAll/ScrollAllRequest.cs Outdated
}
public ScrollAllRequest(Time scrollTime, int numberOfSlices, Field routingField) : this(scrollTime, numberOfSlices)
{
((IScrollAllRequest)this).RoutingField = routingField;
Copy link
Contributor

@russcam russcam Dec 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpick: the cast to interface is not needed

src/Tests/Tests.csproj
<Compile Include="Search\Request\SlicedScrollSearchUsageTests.cs" />
<Compile Include="Search\Request\SearchAfterUsageTests.cs" />
<Compile Include="Search\Scroll\Scroll\SlicedScrollApiTests.cs" />
<Compile Include="Search\Scroll\Scroll\SlicedScrollSearchApiTests.cs" />
Copy link
Contributor

@russcam russcam Dec 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be called ScrollAllApiTests.cs?

Copy link
Member Author

@Mpdreamz Mpdreamz Dec 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No :) These just test the sliced scroll on _search. I've added ScrollAll observable tests to bulkall to save on running times.

Copy link
Contributor

@russcam russcam Dec 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool cool, I thought I must have missed it 👍

@Mpdreamz
ScrollAll()
645c73f 
Exposes a sliced scroll behind an observable with an optional max degree
of parallelism
@Mpdreamz
implemented review feedback from @russcam, unneccessary cast + only s…
43d5e55 
…et scroll to _doc order if the user did not supply a sort of their own
@Mpdreamz Mpdreamz merged commit 1e98d3e into master Dec 7, 2016
Mpdreamz added a commit that referenced this pull request Dec 7, 2016
@Mpdreamz
ScrollAll() (#2443)
987f186 
* ScrollAll()

Exposes a sliced scroll behind an observable with an optional max degree
of parallelism

* implemented review feedback from @russcam, unneccessary cast + only set scroll to _doc order if the user did not supply a sort of their own
@Mpdreamz
Copy link
Member Author

Mpdreamz commented Dec 7, 2016

ported to 5.x

@Mpdreamz Mpdreamz deleted the feature/scroll-all branch December 7, 2016 08:41
@russcam russcam added the v5.0.0 label Dec 13, 2016
awelburn pushed a commit to Artesian/elasticsearch-net that referenced this pull request Nov 6, 2017
@Mpdreamz
ScrollAll() (elastic#2443)
ec8921e 
* ScrollAll()

Exposes a sliced scroll behind an observable with an optional max degree
of parallelism

* implemented review feedback from @russcam, unneccessary cast + only set scroll to _doc order if the user did not supply a sort of their own
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@russcam russcam russcam requested changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Mpdreamz @russcam