Skip to content

feat: add zikade DHT router #10154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed

feat: add zikade DHT router #10154

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ed84e6f
feat: add go-libp2p-kad-dht/v2 DHT router
iand Sep 26, 2023
c9d8582
Rename dhtv2 to dht/zikade
iand Sep 28, 2023
fbc05c1
Ensure Close is called on routers
iand Sep 28, 2023
ac3dd1a
Update boxo import
iand Oct 3, 2023
0398d0c
Remove stray printf
iand Oct 3, 2023
65e9598
Upgrade boxo
iand Oct 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions config/routing.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ func (r *RouterParser) UnmarshalJSON(b []byte) error {
p = &HTTPRouterParams{}
case RouterTypeDHT:
p = &DHTRouterParams{}
case RouterTypeDHTv2:
p = &DHTv2RouterParams{}
case RouterTypeSequential:
p = &ComposableRouterParams{}
case RouterTypeParallel:
Expand All @@ -106,6 +108,7 @@ type RouterType string
const (
RouterTypeHTTP RouterType = "http" // HTTP JSON API for delegated routing systems (IPIP-337).
RouterTypeDHT RouterType = "dht" // DHT router.
RouterTypeDHTv2 RouterType = "dhtv2" // DHT router using go-libp2p-kad-dht/v2.
Copy link
Contributor

@Jorropo Jorropo Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this name is correct, this router is compatible with v1.
So I don't think v2 applies here.

Copy link
Contributor Author

@iand iand Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will allow different custom parameters, which I haven't added yet.

Copy link
Contributor

@Jorropo Jorropo Sep 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean network wise, it's just a different implementation, it's it will talk with other v1 dht servers and clients.
I would name it go-kademlia

The way I understand v2 here is that if I turn this I'm not part of the amino dht anymore.

Copy link
Contributor Author

@iand iand Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see how this name could cause confusion between go-libp2p-kad-dht version 2 and version 2 of the dht protocols. I'll choose a better name - same for boxo

Copy link
Contributor Author

@iand iand Sep 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed router type to dht/zikade

RouterTypeSequential RouterType = "sequential" // Router helper to execute several routers sequentially.
RouterTypeParallel RouterType = "parallel" // Router helper to execute several routers in parallel.
)
Expand Down Expand Up @@ -158,6 +161,11 @@ type DHTRouterParams struct {
PublicIPNetwork bool
}

type DHTv2RouterParams struct {
Mode DHTMode
PublicIPNetwork bool
}

type ComposableRouterParams struct {
Routers []ConfigRouter
Timeout *OptionalDuration `json:",omitempty"`
Expand Down
10 changes: 10 additions & 0 deletions config/routing_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,13 @@ func TestRouterParameters(t *testing.T) {
PublicIPNetwork: false,
},
}},
"router-dhtv2": {Router{
Type: RouterTypeDHTv2,
Parameters: DHTv2RouterParams{
Mode: "auto",
PublicIPNetwork: false,
},
}},
"router-parallel": {
Router{
Type: RouterTypeParallel,
Expand Down Expand Up @@ -97,6 +104,9 @@ func TestRouterParameters(t *testing.T) {
dhtp := r2.Routers["router-dht"].Parameters
require.IsType(&DHTRouterParams{}, dhtp)

dhtv2p := r2.Routers["router-dhtv2"].Parameters
require.IsType(&DHTv2RouterParams{}, dhtv2p)

sp := r2.Routers["router-sequential"].Parameters
require.IsType(&ComposableRouterParams{}, sp)

Expand Down
7 changes: 7 additions & 0 deletions core/node/libp2p/routingopt.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,13 @@ func ConstructDelegatedRouting(routers config.Routers, methods config.Methods, p
Addrs: httpAddrsFromConfig(addrs),
PrivKeyB64: privKey,
},
&irouting.ExtraDHTv2Params{
BootstrapPeers: args.BootstrapPeers,
Host: args.Host,
Validator: args.Validator,
Datastore: args.Datastore,
Context: args.Ctx,
},
)
}
}
Expand Down
46 changes: 27 additions & 19 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
module github.com/ipfs/kubo

// TODO: remove this replacement when https://github.com/ipfs/boxo/pull/472 is merged
// This is https://github.com/iand/boxo/tree/dhtv2
replace github.com/ipfs/boxo => github.com/iand/boxo v0.0.0-20230926144701-607836364a6a

require (
bazil.org/fuse v0.0.0-20200117225306-7b5117fecadc
contrib.go.opencensus.io/exporter/prometheus v0.4.2
Expand All @@ -19,7 +23,7 @@ require (
github.com/ipfs/go-block-format v0.1.2
github.com/ipfs/go-cid v0.4.1
github.com/ipfs/go-cidutil v0.1.0
github.com/ipfs/go-datastore v0.6.0
github.com/ipfs/go-datastore v0.6.1-0.20230901172804-1caa2449ed7c
github.com/ipfs/go-detect-race v0.0.1
github.com/ipfs/go-ds-badger v0.3.0
github.com/ipfs/go-ds-flatfs v0.5.1
Expand Down Expand Up @@ -75,9 +79,9 @@ require (
go.opencensus.io v0.24.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.42.0
go.opentelemetry.io/contrib/propagators/autoprop v0.42.0
go.opentelemetry.io/otel v1.16.0
go.opentelemetry.io/otel/sdk v1.16.0
go.opentelemetry.io/otel/trace v1.16.0
go.opentelemetry.io/otel v1.18.0
go.opentelemetry.io/otel/sdk v1.18.0
go.opentelemetry.io/otel/trace v1.18.0
go.uber.org/dig v1.17.0
go.uber.org/fx v1.20.0
go.uber.org/multierr v1.11.0
Expand All @@ -86,7 +90,7 @@ require (
golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63
golang.org/x/mod v0.12.0
golang.org/x/sync v0.3.0
golang.org/x/sys v0.11.0
golang.org/x/sys v0.12.0
google.golang.org/protobuf v1.31.0
)

Expand Down Expand Up @@ -127,7 +131,7 @@ require (
github.com/google/pprof v0.0.0-20230821062121-407c9e7a662f // indirect
github.com/gorilla/mux v1.8.0 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 // indirect
github.com/hannahhoward/go-pubsub v0.0.0-20200423002714-8d62886cc36e // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
Expand Down Expand Up @@ -157,6 +161,7 @@ require (
github.com/libp2p/go-flow-metrics v0.1.0 // indirect
github.com/libp2p/go-libp2p-asn-util v0.3.0 // indirect
github.com/libp2p/go-libp2p-gostream v0.6.0 // indirect
github.com/libp2p/go-libp2p-kad-dht/v2 v2.0.0-20230925134614-09dd7b0b2f50 // indirect
github.com/libp2p/go-libp2p-xor v0.1.0 // indirect
github.com/libp2p/go-msgio v0.3.0 // indirect
github.com/libp2p/go-nat v0.2.0 // indirect
Expand All @@ -182,8 +187,9 @@ require (
github.com/multiformats/go-varint v0.0.7 // indirect
github.com/onsi/ginkgo/v2 v2.11.0 // indirect
github.com/opencontainers/runtime-spec v1.1.0 // indirect
github.com/openzipkin/zipkin-go v0.4.1 // indirect
github.com/openzipkin/zipkin-go v0.4.2 // indirect
github.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 // indirect
github.com/plprobelab/go-kademlia v0.0.0-20230913171354-443ec1f56080 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/polydawn/refmt v0.89.0 // indirect
github.com/prometheus/client_model v0.4.0 // indirect
Expand Down Expand Up @@ -211,27 +217,29 @@ require (
go.opentelemetry.io/contrib/propagators/b3 v1.17.0 // indirect
go.opentelemetry.io/contrib/propagators/jaeger v1.17.0 // indirect
go.opentelemetry.io/contrib/propagators/ot v1.17.0 // indirect
go.opentelemetry.io/otel/exporters/jaeger v1.14.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.16.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.16.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.14.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.16.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.14.0 // indirect
go.opentelemetry.io/otel/exporters/zipkin v1.14.0 // indirect
go.opentelemetry.io/otel/metric v1.16.0 // indirect
go.opentelemetry.io/proto/otlp v0.19.0 // indirect
go.opentelemetry.io/otel/exporters/jaeger v1.16.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.18.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.18.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.18.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.18.0 // indirect
go.opentelemetry.io/otel/exporters/zipkin v1.18.0 // indirect
go.opentelemetry.io/otel/metric v1.18.0 // indirect
go.opentelemetry.io/otel/sdk/metric v0.41.0 // indirect
go.opentelemetry.io/proto/otlp v1.0.0 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/zap/exp v0.1.0 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/net v0.14.0 // indirect
golang.org/x/oauth2 v0.8.0 // indirect
golang.org/x/oauth2 v0.10.0 // indirect
golang.org/x/term v0.11.0 // indirect
golang.org/x/text v0.12.0 // indirect
golang.org/x/tools v0.12.1-0.20230815132531-74c255bcf846 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
gonum.org/v1/gonum v0.13.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230306155012-7f2fa6fef1f4 // indirect
google.golang.org/grpc v1.55.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20230711160842-782d3b101e98 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 // indirect
google.golang.org/grpc v1.58.0 // indirect
gopkg.in/square/go-jose.v2 v2.5.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Loading