Support for basic authentication

[moscovig]

Adding support for basic authentication credentials

Something like
val solrRDD = new SolrRDD("localhost:2181/solr","collection",username,password,sc,query,rows)

Thanks

[thelabdude]

we're hoping to add this very soon

[weishishuo]

why close this issue? support basic authentication now ?

[kiranchitturi]

yes, basic auth is supported in the latest beta release of spark-solr http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22com.lucidworks.spark%22%20AND%20a%3A%22spark-solr%22

It can be set using system param -Dbasicauth=username:password or -Dsolr.httpclient.config={path_to_config}

Example of config:

httpBasicAuthUser=username
httpBasicAuthPassword=password

Set this on both extraJavaOptions for driver and executor

[noschinl]

What is the reason for using a system parameter instead of making this a parameter of the constructor of the RDD?

As far as I understand it, the current solution has two drawbacks:

• It forces you to either write plain-text password on disk or have it as a parameter to the Java process (and hence visible for all users on the system). Depending on your setup, this is problematic.
• It limits you to a single set of credentials. Host and Collection are parameters of the constructor and hence under control of the application. However, you may use only the host/collection, for which the statically configured credentials (i.e., not under control of the application) are valid. In particular, you cannot use two different collections (or Solr instances) which need different credentials.

[kiranchitturi]

Changing the signature of SolrRDD or SolrRelation is a very major change and it did not seem very practical to do that in our timeframe.

1. System params is the easiest solution for plugging in auth solutions (just like how Solr supports)
2. There is another way to load auth credentials apart from system params. It is kind of an advanced setting

Provide a custom class to spark-solr through which HttpSolrClient can be loaded

• Extend the class https://github.com/LucidWorks/spark-solr/blob/38fdb858e48a6d44571daafc548b3985234fe873/src/main/java/com/lucidworks/spark/util/FusionAuthHttpClient.java and provide a HttpSolrClient.Builder configured with auth credentials
• In the Builder method, you can extend the HttpSolrClient and set authentication headers for every request
• Then load the class into the Spark job application and supply the class name through this system param https://github.com/LucidWorks/spark-solr/blob/38fdb858e48a6d44571daafc548b3985234fe873/src/main/scala/com/lucidworks/spark/util/SolrSupport.scala#L81-L81