librelist archives

Re: [geonode] Error installing GeoNode from source

From:

David Winslow

Date:

2010-11-01 @ 20:58

I have a sufficient version of libcurl and libcurl-devel installed.
 However, now that I look at the output a bit closer, I can see that
setuptools is dumping the output of curl-config (some args) into a call to
gcc and the arguments passed to curl-config are not valid, so gcc is failing
and I guess being ignored since I don't get an error message at the end of
all this.

Anyway, maybe I'll just not use --no-site-packages when setting up my venv
and pretend this problem doesn't exist.  But I'll need to team up with
someone on a system where curl-config works like Pycurl expects to finish
making sure the new Pypi mirror works.

-d

On Mon, Nov 1, 2010 at 4:43 PM, Ariel Nunez <ingenieroariel@gmail.com>wrote:

> I had to upgrade my built-in curl from 7.15 to 7.19 before being able
> to install the latest pycurl.
>
> Maybe if we put an earlier version of pycurl in our pypi mirror? On
> the top of the release notes[1] it seems to imply this is the first
> backwards incompatible release:
>
>
> Version 7.19.0 [requires libcurl-7.19.0 or better]
> --------------
>
>        * Added CURLFILE, ADDRESS_SCOPE and ISSUERCERT options,
>          as well as the APPCONNECT_TIME info.
>
>        * Added PRIMARY_IP info (patch by
>          Yuhui H <eyecat at gmail.com>).
>
>        * Added support for curl_easy_reset through a
>          new 'reset' method on curl objects
>          (patch by Nick Pilon <npilon at oreilly.com>).
>
>        * Added support for OPENSOCKET callbacks.
>          See 'tests/test_opensocket.py' for example
>          usage (patch by Thomas Hunger <teh at camvine.com>).
>
>
> Version 7.18.2
> --------------
>
>        * Added REDIRECT_URL info and M_MAXCONNECTS option
>          (patch by Yuhui H <eyecat at gmail.com>).
>
>        * Added socket_action() method to CurlMulti objects.
>          See 'tests/test_multi_socket_select.py' for example
>          usage (patch by Yuhui H <eyecat at gmail.com>).
>
>        * Added AUTOREFERER option.
>
>        * Allow resetting some list operations (HTTPHEADER,
>          QUOTE, POSTQUOTE, PREQUOTE) by passing an empty
>          list to setopt (patch by Jim Patterson).
>
>
> [1] http://pycurl.sourceforge.net/ChangeLog
>
> On Mon, Nov 1, 2010 at 3:33 PM, David Winslow <dwinslow@opengeo.org>
> wrote:
> > Well, I don't seem to be able to easy_install pycurl in the first place,
> so
> > I'm no longer convinced that just mirroring urlgrabber is sufficient.
> > Maybe this is a Fedora-specific issue, but does anyone who's been able to
> > manually install pycurl so far have some pointers on troubleshooting?
> > $ easy_install pycurl
> > Runs without error, but afterwards trying to install urlgrabber complains
> > that no module named pycurl can be found.  Interestingly, pycurl only
> > produces a curl/ module in my site-packages; it looks like pycurl is the
> > name of the relevant C module.
> > There's a pypi index with just pycurl and urlgrabber at
> > http://dev.geonode.org/pypi/simple/ if you would like to try it out
> > yourself.
> > --
> > David Winslow
> > OpenGeo - http://opengeo.org/
> > On Mon, Nov 1, 2010 at 3:28 PM, Ariel Nunez <ingenieroariel@gmail.com>
> > wrote:
> >>
> >> On Nov 1, 2010, at 14:16, David Winslow <dwinslow@opengeo.org> wrote:
> >>
> >> I was really asking if you knew a better way to handle the problem of
> >> getting stuff into the repository in the first place.
> >>
> >> Gotcha, in that case i think using the --fetch-site-packages option in
> >> compoze looks like the right place to start. I know there are many
> projects
> >> to create PyPi mirrors but have used none of them, Compoze looks good
> enough
> >> to me.
> >>
> >> btw, for whatever reason the place where urlgrabber is failing is using
> >> easy_install to get stuff, so I don't think pip's -f flag is an option
> >> there.
> >> -d
> >> On Mon, Nov 1, 2010 at 3:12 PM, Ariel Nunez <ingenieroariel@gmail.com>
> >> wrote:
> >>>
> >>> Quoting Simon Willinson:
> >>>
> >>> At work, the first line in our requirements.txt file is
> >>> "-f http://pypi.internal/" - which causes pip to use our own internal
> PyPI
> >>> mirror (just an Apache directory listing which pip/setuptools is smart
> >>> enough to scrape).
> >>>
> >>> I dig that idea of just using apache and throwing .tar.gz files there.
> >>>
> >>> Sent from my i
> >>> On Nov 1, 2010, at 13:52, David Winslow <dwinslow@opengeo.org> wrote:
> >>>
> >>> Looking into the "mirror pypi" idea, it seems the Compoze project from
> >>> Repoze is a pretty simple way to set up a Pypi-compatible package index
> with
> >>> packages from Pypi or elsewhere.  I don't think it will automatically
> read
> >>> our pip requirements files, but for now I'll just it to just set up a
> repo
> >>> with these problem packages.
> >>> Does anyone know a better tool for this? (Ariel??) Otherwise I'll take
> a
> >>> look later this week into making a complete mirror with all the Python
> >>> packages we use to prevent this sort of problem from cropping up again.
> >>> --
> >>> David Winslow
> >>> OpenGeo - http://opengeo.org/
> >>>
> >>> On Fri, Oct 29, 2010 at 9:20 AM, Ariel Nunez <ingenieroariel@gmail.com
> >
> >>> wrote:
> >>>>
> >>>> I had to do the same for my install yesterday, in my case it was
> >>>> because the curl version I had was 1.15 and the latest pycurl required
> >>>> 1.17. I had to first install a new curl and then run (inside the
> >>>> venv):
> >>>>
> >>>> pip install pycurl
> >>>> pip install
> >>>> http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz
> >>>>
> >>>> I fully support the idea of setting up our own PyPi mirror, hopefully
> >>>> avoiding the rewrite of the build script.
> >>>>
> >>>> Best,
> >>>> Ariel.
> >>>>
> >>>>
> >>>> On Fri, Oct 29, 2010 at 8:02 AM, David Winslow <dwinslow@opengeo.org>
> >>>> wrote:
> >>>> > I haven't looked very hard but I can't find any information about
> the
> >>>> > removal of urlgrabber from the python package repository.  I think
> it
> >>>> > would
> >>>> > probably be a good idea for us to start mirroring our dependencies
> in
> >>>> > a
> >>>> > repository of our own to avoid this sort of thing.
> >>>> > In the shorter term I would be open to rewriting the build script to
> >>>> > avoid
> >>>> > requiring urlgrabber.
> >>>> > --
> >>>> > David Winslow
> >>>> > OpenGeo - http://opengeo.org/
> >>>> > On Fri, Oct 29, 2010 at 7:56 AM, Christian Spanring
> >>>> > <cspanring@gmail.com>
> >>>> > wrote:
> >>>> >>
> >>>> >> Ole,
> >>>> >>
> >>>> >> the problem is that easy_install can't find urlgrabber at
> >>>> >> pypi.python.org
> >>>> >> anymore, so you have to install urlgrabber manually as workaround:
> >>>> >>
> >>>> >> download and unpack it from http://urlgrabber.baseurl.org/
> >>>> >>
> >>>> >> $ wget
> http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz
> >>>> >> $ tar xvzf urlgrabber-3.9.1.tar.gz
> >>>> >>
> >>>> >> install urlgrabber in GeoNode's Virtual Environment:
> >>>> >>
> >>>> >> $ . bin/activate
> >>>> >> (geonode) $ cd urlgrabber-3.9.1
> >>>> >> (geonode) $ python setup.py install
> >>>> >>
> >>>> >> in my case it asked for pycurl, which I had to install first
> >>>> >> (geonode) $ easy_install pycurl
> >>>> >>
> >>>> >> That should to it.
> >>>> >>
> >>>> >> Christian
> >>>> >>
> >>>> >> On Oct 29, 2010, at 1:54 AM, Ole Nielsen wrote:
> >>>> >>
> >>>> >> > Dear all
> >>>> >> > I am trying to install GeoNode according to the instructions at
> >>>> >> > http://github.com/GeoNode/geonode/blob/1.0-RC2/README.rst and
> have
> >>>> >> > gotten as
> >>>> >> > far as paver build.
> >>>> >> >
> >>>> >> > However, paver build fails with the following error: ImportError:
> >>>> >> > No
> >>>> >> > module named urlgrabber.grabber
> >>>> >> > However, if I start Python normally (i.e. without virtual
> >>>> >> > environment)
> >>>> >> > and try it works just fine:
> >>>> >> >
> >>>> >> > Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
> >>>> >> > [GCC 4.3.3] on linux2
> >>>> >> > Type "help", "copyright", "credits" or "license" for more
> >>>> >> > information.
> >>>> >> > >>> from urlgrabber.grabber import urlgrab, URLGrabError
> >>>> >> > >>>
> >>>> >> >
> >>>> >> > I am running Ubuntu 9.04 - 64 bit.
> >>>> >> >
> >>>> >> > The tail of the paver output follows below.
> >>>> >> > Cheers and thanks for any help
> >>>> >> > Ole Nielsen
> >>>> >> >
> >>>> >> > -- output from paver build --
> >>>> >> >
> >>>> >> >     ########## SUMMARY INFO #########
> >>>> >> >     ################################################
> >>>> >> >     #Attempting install of _rl_accel, sgmlop & pyHnj
> >>>> >> >     #extensions from
> '/tmp/pip-0Buy2p-build/src/rl_addons/rl_accel'
> >>>> >> >     ################################################
> >>>> >> >     ################################################
> >>>> >> >     #Attempting install of _renderPM
> >>>> >> >     #extensions from
> '/tmp/pip-0Buy2p-build/src/rl_addons/renderPM'
> >>>> >> >     # installing with freetype version 21
> >>>> >> >     ################################################
> >>>> >> > Successfully installed avatar gsconfig.py OWSLib reportlab
> >>>> >> > ---> pavement.setup_webapps
> >>>> >> > ---> pavement.setup_geoserver
> >>>> >> > ---> pavement.setup_gs_data
> >>>> >> > Traceback (most recent call last):
> >>>> >> >   File "/home/nielso/sandpit/geonode/bin/paver", line 8, in
> >>>> >> > <module>
> >>>> >> >     load_entry_point('Paver==1.0.1', 'console_scripts',
> 'paver')()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 615, in main
> >>>> >> >     _launch_pavement(args)
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 598, in _launch_pavement
> >>>> >> >     _process_commands(args, auto_pending=auto_pending)
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 563, in _process_commands
> >>>> >> >     task()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 241, in __call__
> >>>> >> >     retval = environment._run_task(self.name, self.needs,
> >>>> >> > self.func)
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 181, in _run_task
> >>>> >> >     return do_task()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 160, in do_task
> >>>> >> >     task()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 241, in __call__
> >>>> >> >     retval = environment._run_task(self.name, self.needs,
> >>>> >> > self.func)
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 181, in _run_task
> >>>> >> >     return do_task()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 160, in do_task
> >>>> >> >     task()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 241, in __call__
> >>>> >> >     retval = environment._run_task(self.name, self.needs,
> >>>> >> > self.func)
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 181, in _run_task
> >>>> >> >     return do_task()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 160, in do_task
> >>>> >> >     task()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 241, in __call__
> >>>> >> >     retval = environment._run_task(self.name, self.needs,
> >>>> >> > self.func)
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 181, in _run_task
> >>>> >> >     return do_task()
> >>>> >> >   File
> >>>> >> >
> >>>> >> >
> 
"/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py",
> >>>> >> > line 161, in do_task
> >>>> >> >     return func(**kw)
> >>>> >> >   File "pavement.py", line 186, in setup_gs_data
> >>>> >> >     grab(src_url, dst_url)
> >>>> >> >   File "pavement.py", line 159, in grab
> >>>> >> >     from urlgrabber.grabber import urlgrab, URLGrabError
> >>>> >> > ImportError: No module named urlgrabber.grabber
> >>>> >> > (geonode)nielso@shiva:~/sandpit/geonode$
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >>
> >>>> >
> >>>> >
> >>>
> >>
> >
> >
>