Skip to content

Commit 5962dd0

Browse files
committed
Add GitLab connector
- Implements GitLab connector using GraphQL API and Work Items API - Fetches projects, issues, merge requests, epics, and releases - Includes pagination support and remote validation - Adds functional tests and unit tests with >92% coverage
1 parent ade544b commit 5962dd0

File tree

16 files changed

+7506
-9
lines changed

16 files changed

+7506
-9
lines changed

.buildkite/pipeline.yml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -599,6 +599,25 @@ steps:
599599
artifact_paths:
600600
- "perf8-report-*/**/*"
601601

602+
- path:
603+
- "connectors/sources/gitlab/**"
604+
- "tests/sources/fixtures/gitlab/**"
605+
- "tests/sources/fixtures/fixture.py"
606+
- "${DOCKERFILE_FTEST_PATH}"
607+
- "requirements/**"
608+
config:
609+
label: "🔨 GitLab"
610+
<<: *test-agents
611+
<<: *retries
612+
env:
613+
- PYTHON_VERSION=3.11
614+
- DATA_SIZE=small
615+
- CONNECTOR=gitlab
616+
command:
617+
- ".buildkite/run_functional_test.sh"
618+
artifact_paths:
619+
- "perf8-report-*/**/*"
620+
602621
# ----
603622
# DRA publishing
604623
# ----

Dockerfile.wolfi

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM docker.elastic.co/wolfi/python:3.11-dev@sha256:2b6277ffc3e044d1457517491642409dbc1f6b35c5c417464cd3e96cc1d26025
1+
FROM docker.elastic.co/wolfi/python:3.11-dev@sha256:c85a849c182a812950c3036b3965426c796813b221b0dd35ae254d6ae6d3f259
22
USER root
33
COPY . /app
44
WORKDIR /app

NOTICE.txt

Lines changed: 86 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1601,6 +1601,32 @@ Apache License
16011601
limitations under the License.
16021602

16031603

1604+
annotated-types
1605+
0.7.0
1606+
MIT License
1607+
The MIT License (MIT)
1608+
1609+
Copyright (c) 2022 the contributors
1610+
1611+
Permission is hereby granted, free of charge, to any person obtaining a copy
1612+
of this software and associated documentation files (the "Software"), to deal
1613+
in the Software without restriction, including without limitation the rights
1614+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
1615+
copies of the Software, and to permit persons to whom the Software is
1616+
furnished to do so, subject to the following conditions:
1617+
1618+
The above copyright notice and this permission notice shall be included in all
1619+
copies or substantial portions of the Software.
1620+
1621+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
1622+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
1623+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
1624+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
1625+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
1626+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
1627+
SOFTWARE.
1628+
1629+
16041630
anyio
16051631
4.11.0
16061632
UNKNOWN
@@ -1854,7 +1880,7 @@ Copyright (C) 2016-present the asyncpg authors and contributors.
18541880

18551881

18561882
attrs
1857-
25.3.0
1883+
25.4.0
18581884
UNKNOWN
18591885
The MIT License (MIT)
18601886

@@ -2606,7 +2632,7 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
26062632

26072633

26082634
cachetools
2609-
6.2.0
2635+
6.2.1
26102636
MIT License
26112637
The MIT License (MIT)
26122638

@@ -5100,11 +5126,11 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
51005126

51015127

51025128
idna
5103-
3.10
5104-
BSD License
5129+
3.11
5130+
UNKNOWN
51055131
BSD 3-Clause License
51065132

5107-
Copyright (c) 2013-2024, Kim Davies and contributors.
5133+
Copyright (c) 2013-2025, Kim Davies and contributors.
51085134
All rights reserved.
51095135

51105136
Redistribution and use in source and binary forms, with or without
@@ -5655,7 +5681,7 @@ MIT License
56555681

56565682

56575683
multidict
5658-
6.6.4
5684+
6.7.0
56595685
Apache License 2.0
56605686
Copyright 2016 Andrew Svetlov and aio-libs contributors
56615687

@@ -5977,7 +6003,7 @@ BSD
59776003
UNKNOWN
59786004

59796005
propcache
5980-
0.4.0
6006+
0.4.1
59816007
Apache Software License
59826008

59836009
Apache License
@@ -6517,6 +6543,58 @@ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
65176543
OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
65186544

65196545

6546+
pydantic
6547+
2.10.6
6548+
MIT License
6549+
The MIT License (MIT)
6550+
6551+
Copyright (c) 2017 to present Pydantic Services Inc. and individual contributors.
6552+
6553+
Permission is hereby granted, free of charge, to any person obtaining a copy
6554+
of this software and associated documentation files (the "Software"), to deal
6555+
in the Software without restriction, including without limitation the rights
6556+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
6557+
copies of the Software, and to permit persons to whom the Software is
6558+
furnished to do so, subject to the following conditions:
6559+
6560+
The above copyright notice and this permission notice shall be included in all
6561+
copies or substantial portions of the Software.
6562+
6563+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
6564+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
6565+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
6566+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
6567+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
6568+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
6569+
SOFTWARE.
6570+
6571+
6572+
pydantic_core
6573+
2.27.2
6574+
MIT License
6575+
The MIT License (MIT)
6576+
6577+
Copyright (c) 2022 Samuel Colvin
6578+
6579+
Permission is hereby granted, free of charge, to any person obtaining a copy
6580+
of this software and associated documentation files (the "Software"), to deal
6581+
in the Software without restriction, including without limitation the rights
6582+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
6583+
copies of the Software, and to permit persons to whom the Software is
6584+
furnished to do so, subject to the following conditions:
6585+
6586+
The above copyright notice and this permission notice shall be included in all
6587+
copies or substantial portions of the Software.
6588+
6589+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
6590+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
6591+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
6592+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
6593+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
6594+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
6595+
SOFTWARE.
6596+
6597+
65206598
pymongo
65216599
4.6.3
65226600
Apache Software License
@@ -8422,7 +8500,7 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI
84228500

84238501

84248502
yarl
8425-
1.21.0
8503+
1.22.0
84268504
Apache Software License
84278505

84288506
Apache License

connectors/config.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,7 @@ def _default_config():
118118
"dir": "connectors.sources.directory:DirectoryDataSource",
119119
"dropbox": "connectors.sources.dropbox:DropboxDataSource",
120120
"github": "connectors.sources.github:GitHubDataSource",
121+
"gitlab": "connectors.sources.gitlab:GitLabDataSource",
121122
"gmail": "connectors.sources.gmail:GMailDataSource",
122123
"google_cloud_storage": "connectors.sources.google_cloud_storage:GoogleCloudStorageDataSource",
123124
"google_drive": "connectors.sources.google_drive:GoogleDriveDataSource",
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#
2+
# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
3+
# or more contributor license agreements. Licensed under the Elastic License 2.0;
4+
# you may not use this file except in compliance with the Elastic License 2.0.
5+
#
6+
"""GitLab connector package.
7+
8+
This package provides integration with GitLab Cloud for syncing projects, issues,
9+
merge requests, epics, releases, and README files to Elasticsearch.
10+
"""
11+
12+
from connectors.sources.gitlab.datasource import GitLabDataSource
13+
14+
__all__ = ["GitLabDataSource"]

0 commit comments

Comments
 (0)