java.net.MalformedURLException: no protocol

[nrv]

When retrieving tweets with the full archive endpoint, asking for all expansions, I have the following error. It does not appear immediately, I'm able to retrieve a bunch of tweets before it is thrown, so it must be linked to a specific tweet/user data. I'm using the 2.0.1 sdk.

com.google.gson.JsonIOException: java.net.MalformedURLException: no protocol:
at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:287)
at com.twitter.clientlib.model.User$CustomTypeAdapterFactory$1.read(User.java:623)
at com.twitter.clientlib.model.User$CustomTypeAdapterFactory$1.read(User.java:612)
at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:285)
at com.twitter.clientlib.model.Expansions$CustomTypeAdapterFactory$1.read(Expansions.java:459)
at com.twitter.clientlib.model.Expansions$CustomTypeAdapterFactory$1.read(Expansions.java:448)
at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:285)
at com.twitter.clientlib.model.Get2TweetsSearchAllResponse$CustomTypeAdapterFactory$1.read(Get2TweetsSearchAllResponse.java:325)
at com.twitter.clientlib.model.Get2TweetsSearchAllResponse$CustomTypeAdapterFactory$1.read(Get2TweetsSearchAllResponse.java:314)
at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
at com.google.gson.Gson.fromJson(Gson.java:991)
at com.google.gson.Gson.fromJson(Gson.java:956)
at com.google.gson.Gson.fromJson(Gson.java:905)
at com.twitter.clientlib.JSON.deserialize(JSON.java:944)
at com.twitter.clientlib.ApiClient.deserialize(ApiClient.java:959)
at com.twitter.clientlib.ApiClient.handleResponse(ApiClient.java:1190)
at com.twitter.clientlib.ApiClient.execute(ApiClient.java:1091)
at com.twitter.clientlib.api.TweetsApi.tweetsFullarchiveSearchWithHttpInfo(TweetsApi.java:3853)
at com.twitter.clientlib.api.TweetsApi.access$6100(TweetsApi.java:87)
at com.twitter.clientlib.api.TweetsApi$APItweetsFullarchiveSearchRequest.execute(TweetsApi.java:4057)

[nrv]

Seems to happen on a witheld account where profile_image_url is empty

{"protected":false,"name":"","created_at":"2011-01-14T09:27:09.000Z","username":"sputnik_fr","entities":{"description":{"urls":[{"start":340,"end":350,"url":"https://support.twitter.com/articles/20169222","expanded_url":"https://support.twitter.com/articles/20169222","display_url":"Learn more"}]}},"public_metrics":{"followers_count":95621,"following_count":219,"tweet_count":236481,"listed_count":1358},"profile_image_url":"","pinned_tweet_id":"1498740347144482820","description":"@sputnik_fr's account has been withheld in Portugal, Finland, Sweden, Ireland, Slovenia, Czech Republic, Poland, Slovakia, Hungary, Italy, Malta, Germany, Greece, Romania, Netherlands, Bulgaria, Austria, Luxembourg, Latvia, United Kingdom, Denmark, Lithuania, Croatia, Estonia, Cyprus, France, Spain, Belgium in response to a legal demand. Learn more.","url":"","id":"238080747","verified":true,"withheld":{"country_codes":["AT","BE","BG","CY","CZ","DE","DK","EE","ES","FI","FR","GB","GR","HR","HU","IE","IT","LT","LU","LV","MT","NL","PL","PT","RO","SE","SI","SK"],"scope":"user"}}

[zacmos]

@nrv, thank you for opening this issue, the fix will be done at the API level and not inside the SDK.
We'll let you know once it's done.

[rzo1]

I am encountering the same issue for the streaming endpoint.

java.net.MalformedURLException: no protocol: 
com.google.gson.JsonIOException: java.net.MalformedURLException: no protocol: 
	at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:287)
	at com.twitter.clientlib.model.User$CustomTypeAdapterFactory$1.read(User.java:623)
	at com.twitter.clientlib.model.User$CustomTypeAdapterFactory$1.read(User.java:612)
	at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
	at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41)
	at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82)
	at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
	at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:285)
	at com.twitter.clientlib.model.Expansions$CustomTypeAdapterFactory$1.read(Expansions.java:459)
	at com.twitter.clientlib.model.Expansions$CustomTypeAdapterFactory$1.read(Expansions.java:448)
	at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
	at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:285)
	at com.twitter.clientlib.model.StreamingTweetResponse$CustomTypeAdapterFactory$1.read(StreamingTweetResponse.java:274)
	at com.twitter.clientlib.model.StreamingTweetResponse$CustomTypeAdapterFactory$1.read(StreamingTweetResponse.java:263)
	at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
	at com.google.gson.Gson.fromJson(Gson.java:991)
	at com.google.gson.Gson.fromJson(Gson.java:956)
	at com.google.gson.Gson.fromJson(Gson.java:905)
	at com.google.gson.Gson.fromJson(Gson.java:876)
	at com.twitter.clientlib.model.StreamingTweetResponse.fromJson(StreamingTweetResponse.java:289)
	at  com.example.TweetsStreamListenersExecutor$TweetsQueuer.queueTweets(TweetsStreamListenersExecutor.java:198)
	at com.example.TweetsStreamListenersExecutor$TweetsQueuer.run(TweetsStreamListenersExecutor.java:179)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.net.MalformedURLException: no protocol: 
	at java.base/java.net.URL.<init>(URL.java:674)
	at java.base/java.net.URL.<init>(URL.java:569)
	at java.base/java.net.URL.<init>(URL.java:516)
	at com.google.gson.internal.bind.TypeAdapters$21.read(TypeAdapters.java:502)
	at com.google.gson.internal.bind.TypeAdapters$21.read(TypeAdapters.java:494)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
	at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:285)
	... 30 more

[zacmos]

@nrv, the API has been updated to handle empty urls.

[noguespi]

not fixed in latest version v2.0.3

com.google.gson.JsonIOException: java.net.MalformedURLException: no protocol: 
	at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:287)
	at com.twitter.clientlib.model.User$CustomTypeAdapterFactory$1.read(User.java:623)
	at com.twitter.clientlib.model.User$CustomTypeAdapterFactory$1.read(User.java:612)
	at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
	at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41)
	at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82)
	at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
	at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:285)
	at com.twitter.clientlib.model.Get2UsersIdFollowersResponse$CustomTypeAdapterFactory$1.read(Get2UsersIdFollowersResponse.java:325)
	at com.twitter.clientlib.model.Get2UsersIdFollowersResponse$CustomTypeAdapterFactory$1.read(Get2UsersIdFollowersResponse.java:314)
	at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
	at com.google.gson.Gson.fromJson(Gson.java:991)
	at com.google.gson.Gson.fromJson(Gson.java:956)
	at com.google.gson.Gson.fromJson(Gson.java:905)
	at com.twitter.clientlib.JSON.deserialize(JSON.java:978)
	at com.twitter.clientlib.ApiClient.deserialize(ApiClient.java:959)
	at com.twitter.clientlib.ApiClient.handleResponse(ApiClient.java:1190)
	at com.twitter.clientlib.ApiClient.execute(ApiClient.java:1091)
	at com.twitter.clientlib.api.UsersApi.usersIdFollowersWithHttpInfo(UsersApi.java:2965)
	at com.twitter.clientlib.api.UsersApi.access$4900(UsersApi.java:77)
	at com.twitter.clientlib.api.UsersApi$APIusersIdFollowersRequest.execute(UsersApi.java:3084)

[ziorufus]

It does not work even when the URL is contained into the field but the protocol is missing.

com.google.gson.JsonIOException: java.net.MalformedURLException: no protocol: Loulasassy.com
        at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:287)
        at com.twitter.clientlib.model.UrlEntity$CustomTypeAdapterFactory$1.read(UrlEntity.java:543)
        at com.twitter.clientlib.model.UrlEntity$CustomTypeAdapterFactory$1.read(UrlEntity.java:532)
        at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
        at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41)
        at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82)
        at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61)
...

I even do not know which parameter I should remove, since I've already deleted profile_image_url. Any idea? Is it possible to fix this in the next version? Or, as an alternative, is it possible to consider the URLs as strings, so that this problem won't happen anymore?

[nrv]

In the meantime, you could still patch the sdk, see : 5533da8 Nicolas Le 09/12/2022 à 15:03, Alessio Palmero Aprosio a écrit :

…

It does not work even when the URL is contained into the field but the protocol is missing. |com.google.gson.JsonIOException: java.net.MalformedURLException: no protocol: Loulasassy.com at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:287) at com.twitter.clientlib.model.UrlEntity$CustomTypeAdapterFactory$1.read(UrlEntity.java:543) at com.twitter.clientlib.model.UrlEntity$CustomTypeAdapterFactory$1.read(UrlEntity.java:532) at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199) at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41) at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82) at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61) ... | I even do not know which parameter should I remove, since I've already deleted |profile_image_url|. Any idea? Is it possible to fix this in the next version? Or, as an alternative, is it possible to consider the URLs as strings, so that this problem won't happen anymore? — Reply to this email directly, view it on GitHub <#30 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACWHXKCNMXKK2HKZIHE4Z3WMM34RANCNFSM522RCKVA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ziorufus]

The problem is that this does not happen only with profile_image_url. In my case, it also happen in old tweets for entities in user fields. I guess that old tweets are not correctly formatted so that the protocol is always provided by the API.

[paul-b-johnson]

I see java.net.MalformedURLException: no protocol: occasionally when using the findUsersById or findUsersByUsername and including the profile_image_url field. I see this issue has been open for a while.

In User, can profileImageUrl be a String rather than a URL? Or can exceptions could be handled so the entire call doesn't fail?