librelist archives

« back to archive

Python & Solr - How to create a results page

Python & Solr - How to create a results page

From:
James Willson
Date:
2012-06-13 @ 14:52
 

Hi,

 

I’m new to this but I am trying to
figure out the best way of achieving this, I'm sure there are a couple of 
ways. My setup is that I have a SOLR search server
and I am using “MySolr” to query it. The SOLR server returns an XML 
response containing
the details of the search result. I need to take this XML response and 
populate a search results page with it which is shown to the user.

 

I have no idea about the best way to do this,
or even where to start. Would anyone be kind enough to give me some advice or
information so I can get started?

 

Thanks!

 

James 		 	   		  

Re: [flask] Python & Solr - How to create a results page

From:
max klymyshyn
Date:
2012-06-13 @ 15:05
That's quite simple.

1) create view which will be your results page, like
@app.route("/search/")
def search():
   # more below...

2) Solr can return JSON which is more convenient because of easy-to-load
dataset (wt=json param), more  here:
http://wiki.apache.org/solr/SolJSON

So you can use json package from python
(json.loads(urllib.urlopen("solrurlandrequest").read()))

3) It's more comfortable to use pysolr (
https://github.com/toastdriven/pysolr ) which handle for you a lot of
boring things related to build requests to Solr

On Wed, Jun 13, 2012 at 5:52 PM, James Willson <james_willson@hotmail.com>wrote:

>  ** **
>
> Hi,****
>
> ** **
>
> I’m new to this but I am trying to figure out the best way of achieving
> this, I'm sure there are a couple of ways. My setup is that I have a SOLR
> search server and I am using “MySolr” to query it. The SOLR server returns
> an XML response containing the details of the search result. I need to take
> this XML response and populate a search results page with it which is shown
> to the user.****
>
> ** **
>
> I have no idea about the best way to do this, or even where to start.
> Would anyone be kind enough to give me some advice or information so I can
> get started?****
>
> ** **
>
> Thanks!****
>
> ** **
>
> James****
>



-- 
Thank you,
Max

Re: [flask] Python & Solr - How to create a results page

From:
Zach Williams
Date:
2012-06-13 @ 15:26
In addition to pysolr, there's also sunburnt:
https://github.com/tow/sunburnt

On Wed, Jun 13, 2012 at 8:05 AM, max klymyshyn <klymyshyn@gmail.com> wrote:

> That's quite simple.
>
> 1) create view which will be your results page, like
> @app.route("/search/")
> def search():
>    # more below...
>
> 2) Solr can return JSON which is more convenient because of easy-to-load
> dataset (wt=json param), more  here:
> http://wiki.apache.org/solr/SolJSON
>
> So you can use json package from python
> (json.loads(urllib.urlopen("solrurlandrequest").read()))
>
> 3) It's more comfortable to use pysolr (
> https://github.com/toastdriven/pysolr ) which handle for you a lot of
> boring things related to build requests to Solr
>
> On Wed, Jun 13, 2012 at 5:52 PM, James Willson <james_willson@hotmail.com>wrote:
>
>>  ** **
>>
>> Hi,****
>>
>> ** **
>>
>> I’m new to this but I am trying to figure out the best way of achieving
>> this, I'm sure there are a couple of ways. My setup is that I have a SOLR
>> search server and I am using “MySolr” to query it. The SOLR server returns
>> an XML response containing the details of the search result. I need to take
>> this XML response and populate a search results page with it which is shown
>> to the user.****
>>
>> ** **
>>
>> I have no idea about the best way to do this, or even where to start.
>> Would anyone be kind enough to give me some advice or information so I can
>> get started?****
>>
>> ** **
>>
>> Thanks!****
>>
>> ** **
>>
>> James****
>>
>
>
>
> --
> Thank you,
> Max
>
>

Re: [flask] Python & Solr - How to create a results page

From:
Igor Davydenko
Date:
2012-06-13 @ 15:02
Hi,

Take a look on pysolr library, https://github.com/toastdriven/pysolr/

This library is really easy wrapper for getting data from solr, all
you need is connect to your Solr instance with
``pysolr.Solr(SERVER_URL)`` and then you'll able to perform
``search``, ``delete`` and other queries.

On 13 June 2012 17:52, James Willson <james_willson@hotmail.com> wrote:
>
>
> Hi,
>
>
>
> I’m new to this but I am trying to figure out the best way of achieving
> this, I'm sure there are a couple of ways. My setup is that I have a SOLR
> search server and I am using “MySolr” to query it. The SOLR server returns
> an XML response containing the details of the search result. I need to take
> this XML response and populate a search results page with it which is shown
> to the user.
>
>
>
> I have no idea about the best way to do this, or even where to start. Would
> anyone be kind enough to give me some advice or information so I can get
> started?
>
>
>
> Thanks!
>
>
>
> James



-- 
Sincerely,
Igor Davydenko

Re: [flask] Python & Solr - How to create a results page

From:
James Willson
Date:
2012-06-14 @ 08:32
Ok, im going the JSON route with an XML fallback. I was convinced when I 
did some more reading. These are the steps that I see.
1. Client submits query to Python code through a GET requiest2. Python 
code works out if client has javascript avaliabile 3. If Javascript is 
availiable, Solr server returns JSON result, if not, renders the page 
server side using an XML response.4. Page results populated entierly by 
Ajax Javascript

Date: Wed, 13 Jun 2012 10:09:38 -0700
From: hey@zachwill.com
Subject: Re: [flask] Python & Solr - How to create a results page
To: flask@librelist.com

I'm really anti-XML in all of my own code — feel it's slower to parse and 
use in other languages' data structures than JSON (which is very similar 
to Python's dictionaries). I'm sure there's valid use cases for it, but I 
would try to go the JSON route with your application (I might just be 
bikeshedding though). Would love to hear other's opinions, though.



On Wed, Jun 13, 2012 at 9:02 AM, James Willson <james_willson@hotmail.com> wrote:






Hi Zack, thank you for the reply. In your opinion, does the benefit of 
having it delivered by JSON if JS is enabled warrent the additional 
complexity of coding in an XML fallback, rather than going all XML?



Date: Wed, 13 Jun 2012 08:47:27 -0700
From: hey@zachwill.com
Subject: Re: [flask] Python & Solr - How to create a results page


To: flask@librelist.com

Just a quick heads up on the JSON/JavaScript thing — you can check the 
X-Requested-With header is XMLHttpRequest or see if it has a "callback" 
request.args to send back JSON/JSONP, and if not, send back a search 
results HTML page.





On Wed, Jun 13, 2012 at 8:30 AM, James Willson <james_willson@hotmail.com> wrote:








Hi,
Thank you for your responses. I really appreciate it. Id like to reply to 
the main points.
I've actually spent about 10 hours on an hourly rate working with the 
creator of MySolr to get it integrated and running, and from what I can 
see it works quite nicely. I liked the way that MySolr worked, is it worth
me changing now that I have MySolr setup? I mean, are there any 
disadntages to MySolr?




Regarding Json, I did know about this, but not only is XML the default, 
but also JSON requires javascript, so if I have a client who is not using 
javascript the search service will be broken. I'm not sure I wanted that, 
even though most people use javascript.




Finally, regarding the example code, I understand that is the start of a 
view page, but what I am unsure about is combining the Python page with 
the XML results. How do I actually get the XML response into the page?




Again, I hugely appreciate the information.
Date: Wed, 13 Jun 2012 18:05:01 +0300
From: klymyshyn@gmail.com



Subject: Re: [flask] Python & Solr - How to create a results page

To: flask@librelist.com

That's quite simple.
1) create view which will be your results page, like@app.route("/search/")



def search():   # more below...
2) Solr can return JSON which is more convenient because of easy-to-load 
dataset (wt=json param), more  here:
http://wiki.apache.org/solr/SolJSON
So you can use json package from python 
(json.loads(urllib.urlopen("solrurlandrequest").read()))





3) It's more comfortable to use pysolr 
(https://github.com/toastdriven/pysolr ) which handle for you a lot of 
boring things related to build requests to Solr





On Wed, Jun 13, 2012 at 5:52 PM, James Willson <james_willson@hotmail.com> wrote:









 


Hi,


 


I’m new to this but I am trying to
figure out the best way of achieving this, I'm sure there are a couple of 
ways. My setup is that I have a SOLR search server
and I am using “MySolr” to query it. The SOLR server returns an XML 
response containing
the details of the search result. I need to take this XML response and 
populate a search results page with it which is shown to the user.


 


I have no idea about the best way to do this,
or even where to start. Would anyone be kind enough to give me some advice or
information so I can get started?


 


Thanks!


 


James
 		 	   		  


-- 
Thank you,Max

 		 	   		  

 		 	   		  

 		 	   		  

Re: [flask] Python & Solr - How to create a results page

From:
Anthony Ford
Date:
2012-06-14 @ 14:50
@James, I'm not sure I understand why you're saying that your client needs
to have JS installed/activated within their browser. Are you not handling
the MySolr response server side? If you are handling it server-side, then
your clients won't need JS, as you could then parse the JSON input via a
json library in python (such as simplejson), and then treating the data as
a normal python dict, and using Jinja to generate your search results.

Anthony Ford,
KF5IBN,
Research Assistant,
Center for Advanced Radio Astronomy,
University of Texas at Brownsville
ford.anthonyj@gmail.com



On Thu, Jun 14, 2012 at 4:32 AM, James Willson <james_willson@hotmail.com>wrote:

>  Ok, im going the JSON route with an XML fallback. I was convinced when I
> did some more reading. These are the steps that I see.
>
> 1. Client submits query to Python code through a GET requiest
> 2. Python code works out if client has javascript avaliabile
> 3. If Javascript is availiable, Solr server returns JSON result, if not,
> renders the page server side using an XML response.
> 4. Page results populated entierly by Ajax Javascript
>
>
> ------------------------------
> Date: Wed, 13 Jun 2012 10:09:38 -0700
>
> From: hey@zachwill.com
> Subject: Re: [flask] Python & Solr - How to create a results page
> To: flask@librelist.com
>
> I'm really anti-XML in all of my own code — feel it's slower to parse and
> use in other languages' data structures than JSON (which is very similar to
> Python's dictionaries). I'm sure there's valid use cases for it, but I
> would try to go the JSON route with your application (I might just be
> bikeshedding <http://blue.bikeshed.org/> though). Would love to hear
> other's opinions, though.
>
> On Wed, Jun 13, 2012 at 9:02 AM, James Willson <james_willson@hotmail.com>wrote:
>
>  Hi Zack, thank you for the reply. In your opinion, does the benefit of
> having it delivered by JSON if JS is enabled warrent the additional
> complexity of coding in an XML fallback, rather than going all XML?
>
> ------------------------------
> Date: Wed, 13 Jun 2012 08:47:27 -0700
> From: hey@zachwill.com
>
> Subject: Re: [flask] Python & Solr - How to create a results page
> To: flask@librelist.com
>
> Just a quick heads up on the JSON/JavaScript thing — you can check the
> X-Requested-With header is XMLHttpRequest or see if it has a "callback"
> request.args to send back JSON/JSONP, and if not, send back a search
> results HTML page.
>
> On Wed, Jun 13, 2012 at 8:30 AM, James Willson <james_willson@hotmail.com>wrote:
>
>  Hi,
>
> Thank you for your responses. I really appreciate it. Id like to reply to
> the main points.
>
> I've actually spent about 10 hours on an hourly rate working with the
> creator of MySolr to get it integrated and running, and from what I can see
> it works quite nicely. I liked the way that MySolr worked, is it worth me
> changing now that I have MySolr setup? I mean, are there any disadntages to
> MySolr?
>
> Regarding Json, I did know about this, but not only is XML the default,
> but also JSON requires javascript, so if I have a client who is not using
> javascript the search service will be broken. I'm not sure I wanted that,
> even though most people use javascript.
>
> Finally, regarding the example code, I understand that is the start of a
> view page, but what I am unsure about is combining the Python page with the
> XML results. How do I actually get the XML response into the page?
>
> Again, I hugely appreciate the information.
>
> ------------------------------
> Date: Wed, 13 Jun 2012 18:05:01 +0300
> From: klymyshyn@gmail.com
> Subject: Re: [flask] Python & Solr - How to create a results page
> To: flask@librelist.com
>
>
> That's quite simple.
>
> 1) create view which will be your results page, like
> @app.route("/search/")
> def search():
>    # more below...
>
> 2) Solr can return JSON which is more convenient because of easy-to-load
> dataset (wt=json param), more  here:
> http://wiki.apache.org/solr/SolJSON
>
> So you can use json package from python
> (json.loads(urllib.urlopen("solrurlandrequest").read()))
>
> 3) It's more comfortable to use pysolr (
> https://github.com/toastdriven/pysolr ) which handle for you a lot of
> boring things related to build requests to Solr
>
> On Wed, Jun 13, 2012 at 5:52 PM, James Willson <james_willson@hotmail.com>wrote:
>
>  ** **
> Hi,****
> ** **
> I’m new to this but I am trying to figure out the best way of achieving
> this, I'm sure there are a couple of ways. My setup is that I have a SOLR
> search server and I am using “MySolr” to query it. The SOLR server returns
> an XML response containing the details of the search result. I need to take
> this XML response and populate a search results page with it which is shown
> to the user.****
> ** **
> I have no idea about the best way to do this, or even where to start.
> Would anyone be kind enough to give me some advice or information so I can
> get started?****
> ** **
> Thanks!****
> ** **
> James****
>
>
>
>
> --
> Thank you,
> Max
>
>
>
>

Re: [flask] Python & Solr - How to create a results page

From:
James Willson
Date:
2012-06-13 @ 15:30
Hi,
Thank you for your responses. I really appreciate it. Id like to reply to 
the main points.
I've actually spent about 10 hours on an hourly rate working with the 
creator of MySolr to get it integrated and running, and from what I can 
see it works quite nicely. I liked the way that MySolr worked, is it worth
me changing now that I have MySolr setup? I mean, are there any 
disadntages to MySolr?
Regarding Json, I did know about this, but not only is XML the default, 
but also JSON requires javascript, so if I have a client who is not using 
javascript the search service will be broken. I'm not sure I wanted that, 
even though most people use javascript.
Finally, regarding the example code, I understand that is the start of a 
view page, but what I am unsure about is combining the Python page with 
the XML results. How do I actually get the XML response into the page?
Again, I hugely appreciate the information.
Date: Wed, 13 Jun 2012 18:05:01 +0300
From: klymyshyn@gmail.com
Subject: Re: [flask] Python & Solr - How to create a results page
To: flask@librelist.com

That's quite simple.
1) create view which will be your results page, 
like@app.route("/search/")def search():   # more below...
2) Solr can return JSON which is more convenient because of easy-to-load 
dataset (wt=json param), more  here:
http://wiki.apache.org/solr/SolJSON
So you can use json package from python 
(json.loads(urllib.urlopen("solrurlandrequest").read()))

3) It's more comfortable to use pysolr 
(https://github.com/toastdriven/pysolr ) which handle for you a lot of 
boring things related to build requests to Solr

On Wed, Jun 13, 2012 at 5:52 PM, James Willson <james_willson@hotmail.com> wrote:





 

Hi,

 

I’m new to this but I am trying to
figure out the best way of achieving this, I'm sure there are a couple of 
ways. My setup is that I have a SOLR search server
and I am using “MySolr” to query it. The SOLR server returns an XML 
response containing
the details of the search result. I need to take this XML response and 
populate a search results page with it which is shown to the user.

 

I have no idea about the best way to do this,
or even where to start. Would anyone be kind enough to give me some advice or
information so I can get started?

 

Thanks!

 

James 		 	   		  


-- 
Thank you,Max

 		 	   		  

Re: [flask] Python & Solr - How to create a results page

From:
Zach Williams
Date:
2012-06-13 @ 15:47
Just a quick heads up on the JSON/JavaScript thing — you can check the
X-Requested-With header is XMLHttpRequest or see if it has a "callback"
request.args to send back JSON/JSONP, and if not, send back a search
results HTML page.

On Wed, Jun 13, 2012 at 8:30 AM, James Willson <james_willson@hotmail.com>wrote:

>  Hi,
>
> Thank you for your responses. I really appreciate it. Id like to reply to
> the main points.
>
> I've actually spent about 10 hours on an hourly rate working with the
> creator of MySolr to get it integrated and running, and from what I can see
> it works quite nicely. I liked the way that MySolr worked, is it worth me
> changing now that I have MySolr setup? I mean, are there any disadntages to
> MySolr?
>
> Regarding Json, I did know about this, but not only is XML the default,
> but also JSON requires javascript, so if I have a client who is not using
> javascript the search service will be broken. I'm not sure I wanted that,
> even though most people use javascript.
>
> Finally, regarding the example code, I understand that is the start of a
> view page, but what I am unsure about is combining the Python page with the
> XML results. How do I actually get the XML response into the page?
>
> Again, I hugely appreciate the information.
>
> ------------------------------
> Date: Wed, 13 Jun 2012 18:05:01 +0300
> From: klymyshyn@gmail.com
> Subject: Re: [flask] Python & Solr - How to create a results page
> To: flask@librelist.com
>
>
> That's quite simple.
>
> 1) create view which will be your results page, like
> @app.route("/search/")
> def search():
>    # more below...
>
> 2) Solr can return JSON which is more convenient because of easy-to-load
> dataset (wt=json param), more  here:
> http://wiki.apache.org/solr/SolJSON
>
> So you can use json package from python
> (json.loads(urllib.urlopen("solrurlandrequest").read()))
>
> 3) It's more comfortable to use pysolr (
> https://github.com/toastdriven/pysolr ) which handle for you a lot of
> boring things related to build requests to Solr
>
> On Wed, Jun 13, 2012 at 5:52 PM, James Willson <james_willson@hotmail.com>wrote:
>
>  ** **
> Hi,****
> ** **
> I’m new to this but I am trying to figure out the best way of achieving
> this, I'm sure there are a couple of ways. My setup is that I have a SOLR
> search server and I am using “MySolr” to query it. The SOLR server returns
> an XML response containing the details of the search result. I need to take
> this XML response and populate a search results page with it which is shown
> to the user.****
> ** **
> I have no idea about the best way to do this, or even where to start.
> Would anyone be kind enough to give me some advice or information so I can
> get started?****
> ** **
> Thanks!****
> ** **
> James****
>
>
>
>
> --
> Thank you,
> Max
>
>

Re: [flask] Python & Solr - How to create a results page

From:
James Willson
Date:
2012-06-13 @ 16:02
Hi Zack, thank you for the reply. In your opinion, does the benefit of 
having it delivered by JSON if JS is enabled warrent the additional 
complexity of coding in an XML fallback, rather than going all XML?

Date: Wed, 13 Jun 2012 08:47:27 -0700
From: hey@zachwill.com
Subject: Re: [flask] Python & Solr - How to create a results page
To: flask@librelist.com

Just a quick heads up on the JSON/JavaScript thing — you can check the 
X-Requested-With header is XMLHttpRequest or see if it has a "callback" 
request.args to send back JSON/JSONP, and if not, send back a search 
results HTML page.



On Wed, Jun 13, 2012 at 8:30 AM, James Willson <james_willson@hotmail.com> wrote:






Hi,
Thank you for your responses. I really appreciate it. Id like to reply to 
the main points.
I've actually spent about 10 hours on an hourly rate working with the 
creator of MySolr to get it integrated and running, and from what I can 
see it works quite nicely. I liked the way that MySolr worked, is it worth
me changing now that I have MySolr setup? I mean, are there any 
disadntages to MySolr?


Regarding Json, I did know about this, but not only is XML the default, 
but also JSON requires javascript, so if I have a client who is not using 
javascript the search service will be broken. I'm not sure I wanted that, 
even though most people use javascript.


Finally, regarding the example code, I understand that is the start of a 
view page, but what I am unsure about is combining the Python page with 
the XML results. How do I actually get the XML response into the page?


Again, I hugely appreciate the information.
Date: Wed, 13 Jun 2012 18:05:01 +0300
From: klymyshyn@gmail.com

Subject: Re: [flask] Python & Solr - How to create a results page

To: flask@librelist.com

That's quite simple.
1) create view which will be your results page, like@app.route("/search/")

def search():   # more below...
2) Solr can return JSON which is more convenient because of easy-to-load 
dataset (wt=json param), more  here:
http://wiki.apache.org/solr/SolJSON
So you can use json package from python 
(json.loads(urllib.urlopen("solrurlandrequest").read()))



3) It's more comfortable to use pysolr 
(https://github.com/toastdriven/pysolr ) which handle for you a lot of 
boring things related to build requests to Solr



On Wed, Jun 13, 2012 at 5:52 PM, James Willson <james_willson@hotmail.com> wrote:







 


Hi,


 


I’m new to this but I am trying to
figure out the best way of achieving this, I'm sure there are a couple of 
ways. My setup is that I have a SOLR search server
and I am using “MySolr” to query it. The SOLR server returns an XML 
response containing
the details of the search result. I need to take this XML response and 
populate a search results page with it which is shown to the user.


 


I have no idea about the best way to do this,
or even where to start. Would anyone be kind enough to give me some advice or
information so I can get started?


 


Thanks!


 


James
 		 	   		  


-- 
Thank you,Max

 		 	   		  

 		 	   		  

Re: [flask] Python & Solr - How to create a results page

From:
Zach Williams
Date:
2012-06-13 @ 17:09
I'm really anti-XML in all of my own code — feel it's slower to parse and
use in other languages' data structures than JSON (which is very similar to
Python's dictionaries). I'm sure there's valid use cases for it, but I
would try to go the JSON route with your application (I might just be
bikeshedding <http://blue.bikeshed.org/> though). Would love to hear
other's opinions, though.

On Wed, Jun 13, 2012 at 9:02 AM, James Willson <james_willson@hotmail.com>wrote:

>  Hi Zack, thank you for the reply. In your opinion, does the benefit of
> having it delivered by JSON if JS is enabled warrent the additional
> complexity of coding in an XML fallback, rather than going all XML?
>
> ------------------------------
> Date: Wed, 13 Jun 2012 08:47:27 -0700
> From: hey@zachwill.com
>
> Subject: Re: [flask] Python & Solr - How to create a results page
> To: flask@librelist.com
>
> Just a quick heads up on the JSON/JavaScript thing — you can check the
> X-Requested-With header is XMLHttpRequest or see if it has a "callback"
> request.args to send back JSON/JSONP, and if not, send back a search
> results HTML page.
>
> On Wed, Jun 13, 2012 at 8:30 AM, James Willson <james_willson@hotmail.com>wrote:
>
>  Hi,
>
> Thank you for your responses. I really appreciate it. Id like to reply to
> the main points.
>
> I've actually spent about 10 hours on an hourly rate working with the
> creator of MySolr to get it integrated and running, and from what I can see
> it works quite nicely. I liked the way that MySolr worked, is it worth me
> changing now that I have MySolr setup? I mean, are there any disadntages to
> MySolr?
>
> Regarding Json, I did know about this, but not only is XML the default,
> but also JSON requires javascript, so if I have a client who is not using
> javascript the search service will be broken. I'm not sure I wanted that,
> even though most people use javascript.
>
> Finally, regarding the example code, I understand that is the start of a
> view page, but what I am unsure about is combining the Python page with the
> XML results. How do I actually get the XML response into the page?
>
> Again, I hugely appreciate the information.
>
> ------------------------------
> Date: Wed, 13 Jun 2012 18:05:01 +0300
> From: klymyshyn@gmail.com
> Subject: Re: [flask] Python & Solr - How to create a results page
> To: flask@librelist.com
>
>
> That's quite simple.
>
> 1) create view which will be your results page, like
> @app.route("/search/")
> def search():
>    # more below...
>
> 2) Solr can return JSON which is more convenient because of easy-to-load
> dataset (wt=json param), more  here:
> http://wiki.apache.org/solr/SolJSON
>
> So you can use json package from python
> (json.loads(urllib.urlopen("solrurlandrequest").read()))
>
> 3) It's more comfortable to use pysolr (
> https://github.com/toastdriven/pysolr ) which handle for you a lot of
> boring things related to build requests to Solr
>
> On Wed, Jun 13, 2012 at 5:52 PM, James Willson <james_willson@hotmail.com>wrote:
>
>  ** **
> Hi,****
> ** **
> I’m new to this but I am trying to figure out the best way of achieving
> this, I'm sure there are a couple of ways. My setup is that I have a SOLR
> search server and I am using “MySolr” to query it. The SOLR server returns
> an XML response containing the details of the search result. I need to take
> this XML response and populate a search results page with it which is shown
> to the user.****
> ** **
> I have no idea about the best way to do this, or even where to start.
> Would anyone be kind enough to give me some advice or information so I can
> get started?****
> ** **
> Thanks!****
> ** **
> James****
>
>
>
>
> --
> Thank you,
> Max
>
>
>