librelist archives

« back to archive

Installation problems via pip

Installation problems via pip

From:
xancorreu
Date:
2010-10-18 @ 19:00
Hi,

I installed flask via pip:

# pip install flask
Downloading/unpacking flask
  Running setup.py egg_info for package flask
    warning: no previously-included files matching '*.pyc' found under 
directory 'docs'
    warning: no previously-included files matching '*.pyo' found under 
directory 'docs'
    warning: no previously-included files matching '*.pyc' found under 
directory 'tests'
    warning: no previously-included files matching '*.pyo' found under 
directory 'tests'
    warning: no previously-included files matching '*.pyc' found under 
directory 'examples'
    warning: no previously-included files matching '*.pyo' found under 
directory 'examples'
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'docs/_themes/.git'
Downloading/unpacking Werkzeug>=0.6.1 (from flask)
  Could not fetch URL http://trac.pocoo.org/repos/werkzeug/trunk (from 
http://pypi.python.org/simple/Werkzeug/): HTTP Error 404: Not Found
  Will skip URL http://trac.pocoo.org/repos/werkzeug/trunk when looking 
for download links for Werkzeug>=0.6.1 (from flask)
  Downloading Werkzeug-0.6.2.tar.gz (1.7Mb): 1.7Mb downloaded
  Running setup.py egg_info for package Werkzeug
    no previously-included directories found matching 'docs/_build/doctrees'
Downloading/unpacking Jinja2>=2.4 (from flask)
  Downloading Jinja2-2.5.2.tar.gz (697Kb): 697Kb downloaded
  Running setup.py egg_info for package Jinja2
    warning: no previously-included files matching '*' found under 
directory 'docs/_build/doctrees'
    warning: no previously-included files matching '*.pyc' found under 
directory 'jinja2'
    warning: no previously-included files matching '*.pyc' found under 
directory 'docs'
    warning: no previously-included files matching '*.pyo' found under 
directory 'jinja2'
    warning: no previously-included files matching '*.pyo' found under 
directory 'docs'
Installing collected packages: flask, Jinja2, Werkzeug
  Running setup.py install for flask
    warning: no previously-included files matching '*.pyc' found under 
directory 'docs'
    warning: no previously-included files matching '*.pyo' found under 
directory 'docs'
    warning: no previously-included files matching '*.pyc' found under 
directory 'tests'
    warning: no previously-included files matching '*.pyo' found under 
directory 'tests'
    warning: no previously-included files matching '*.pyc' found under 
directory 'examples'
    warning: no previously-included files matching '*.pyo' found under 
directory 'examples'
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'docs/_themes/.git'
  Running setup.py install for Jinja2
    warning: no previously-included files matching '*' found under 
directory 'docs/_build/doctrees'
    warning: no previously-included files matching '*.pyc' found under 
directory 'jinja2'
    warning: no previously-included files matching '*.pyc' found under 
directory 'docs'
    warning: no previously-included files matching '*.pyo' found under 
directory 'jinja2'
    warning: no previously-included files matching '*.pyo' found under 
directory 'docs'
  Running setup.py install for Werkzeug
    no previously-included directories found matching 'docs/_build/doctrees'
Successfully installed flask


But when I run python prova.py I return

xan@negre:~/proves/flask$ python prova.py
Traceback (most recent call last):
  File "prova.py", line 1, in <module>
    from flask import Flask
ImportError: No module named flask


prova.py simply is:
from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return "Hello World!"

if __name__ == '__main__':
    app.run()

What happens?

Thanks a lot,
Xan.

Re: [flask] Installation problems via pip

From:
Jonas Galvez
Date:
2010-10-19 @ 00:21
I'm really not sure, but I'd guess "pip" might be running a version of
Python different from the one you get when you type "python".

You can save yourself all this trouble by learning how modules are
loaded in Python (which is really simple[1]), and keep your own /lib
sub-directory in your projects. I myself have a main "stdlib"
repository with the most common packages I use in Python, and I
manually copy the relevant sub-directories to an specific project's
/lib directory when I need something.

Let me try and walk you through it. First, create your own library directory:

mkdir -p ~/stdlib/python

Then download Flask into it:

cd ~/stdlib/python
wget http://nodeload.github.com/mitsuhiko/flask/tarball/0.6
tar zxvf mitsuhiko-flask-0.6-0-g5cadd9d.tar.gz
mv mitsuhiko-flask-5cadd9d/ flask-0.6.0/

Now copy the library directory to your project's lib directory:

cd ~/src/yourproject
mkdir lib
cp -r ~/stdlib/python/flask-0.6-0/flask lib/flask

So from this point on, all you have to do is add the lib/ subdirectory
to the PYTHON_PATH before you import the modules in your app. I do
that by having a support.py file. The support.py files adds '/lib' to
the PYTHON_PATH and loads all dependencies. The first few lines of the
file look like this:

import sys
from os import path
APP_ROOT = path.abspath(path.join(path.dirname(__file__)))
sys.path.append(path.join(APP_ROOT, 'lib'))
from flask import *
# other library imports

Then, in app.py, I just add the following line at the beginning to load it all:

from support import *

But sure, using (pip|easy_install|etc) is 'easier' (when it works) :)

-- Jonas

[1] http://docs.python.org/tutorial/modules.html

On Mon, Oct 18, 2010 at 5:00 PM, xancorreu <xancorreu@gmail.com> wrote:
> I installed flask via pip:
> # pip install flask
> Successfully installed flask
> But when I run python prova.py I return
> xan@negre:~/proves/flask$ python prova.py
> Traceback (most recent call last):
>  File "prova.py", line 1, in <module>
>    from flask import Flask
> ImportError: No module named flask
> What happens?
> Thanks a lot,
> Xan.

Re: [flask] Installation problems via pip

From:
Jonathan Chen
Date:
2010-10-19 @ 19:47
Yeah, but programming in perl still gives me the shivers.

Jonathan C.
------Original Message------
From: danjac354@gmail.com
Sender: flask@librelist.com
To: flask@librelist.com
ReplyTo: flask@librelist.com
Subject: Re: [flask] Installation problems via pip
Sent: Oct 19, 2010 12:09 PM

> I feel your pain.  You want python to work like a standard unix
> program, where you say "Look here first for your data."  In Perl it
> was as simple as:
>
> http://perldoc.perl.org/lib.html
>
> #!/usr/bin/env perl
>
> # The wheel is faulty! I have re-invented it here!
> # Look upon my works ye mighty, and despair!
>
> use lib 'Wheel2';

Whenever there's a new feature in Python or Ruby you find Perl has had
it for years....

Re: [flask] Installation problems via pip

From:
kevin beckford
Date:
2010-10-19 @ 22:22
> Yeah, but programming in perl still gives me the shivers.

I once typed the exact same words about awk.  ;)

Re: [flask] Installation problems via pip

From:
Dag Odenhall
Date:
2010-10-19 @ 22:59
On Tue, 2010-10-19 at 15:22 -0700, kevin beckford wrote:
> > Yeah, but programming in perl still gives me the shivers.
> 
> I once typed the exact same words about awk.  ;)

AWK is awkward and Perl leads to perlonged suffering.

Re: [flask] Installation problems via pip

From:
xancorreu
Date:
2010-10-19 @ 17:07
Wow!. Very big job for running microframework. Could it be as easy as 
bottle: download and run ;-)

I will try it and soon I reply....

Xan.

En/na Jonas Galvez ha escrit:
> I'm really not sure, but I'd guess "pip" might be running a version of
> Python different from the one you get when you type "python".
>
> You can save yourself all this trouble by learning how modules are
> loaded in Python (which is really simple[1]), and keep your own /lib
> sub-directory in your projects. I myself have a main "stdlib"
> repository with the most common packages I use in Python, and I
> manually copy the relevant sub-directories to an specific project's
> /lib directory when I need something.
>
> Let me try and walk you through it. First, create your own library directory:
>
> mkdir -p ~/stdlib/python
>
> Then download Flask into it:
>
> cd ~/stdlib/python
> wget http://nodeload.github.com/mitsuhiko/flask/tarball/0.6
> tar zxvf mitsuhiko-flask-0.6-0-g5cadd9d.tar.gz
> mv mitsuhiko-flask-5cadd9d/ flask-0.6.0/
>
> Now copy the library directory to your project's lib directory:
>
> cd ~/src/yourproject
> mkdir lib
> cp -r ~/stdlib/python/flask-0.6-0/flask lib/flask
>
> So from this point on, all you have to do is add the lib/ subdirectory
> to the PYTHON_PATH before you import the modules in your app. I do
> that by having a support.py file. The support.py files adds '/lib' to
> the PYTHON_PATH and loads all dependencies. The first few lines of the
> file look like this:
>
> import sys
> from os import path
> APP_ROOT = path.abspath(path.join(path.dirname(__file__)))
> sys.path.append(path.join(APP_ROOT, 'lib'))
> from flask import *
> # other library imports
>
> Then, in app.py, I just add the following line at the beginning to load it all:
>
> from support import *
>
> But sure, using (pip|easy_install|etc) is 'easier' (when it works) :)
>
> -- Jonas
>
> [1] http://docs.python.org/tutorial/modules.html
>
> On Mon, Oct 18, 2010 at 5:00 PM, xancorreu <xancorreu@gmail.com> wrote:
>   
>> I installed flask via pip:
>> # pip install flask
>> Successfully installed flask
>> But when I run python prova.py I return
>> xan@negre:~/proves/flask$ python prova.py
>> Traceback (most recent call last):
>>  File "prova.py", line 1, in <module>
>>    from flask import Flask
>> ImportError: No module named flask
>> What happens?
>> Thanks a lot,
>> Xan.
>>     

Re: [flask] Installation problems via pip

From:
danjac354@gmail.com
Date:
2010-10-19 @ 17:16
On 19 October 2010 18:07, xancorreu <xancorreu@gmail.com> wrote:
> Wow!. Very big job for running microframework. Could it be as easy as
> bottle: download and run ;-)
>

Well, to a point. Until you actually need to do some real work, in
which case you'll need at least some libraries, and you'll run into
the same problem you'd have anyway.

Re: [flask] Installation problems via pip

From:
Jonas Galvez
Date:
2010-10-19 @ 17:12
On Tue, Oct 19, 2010 at 3:07 PM, xancorreu <xancorreu@gmail.com> wrote:
> Wow!. Very big job for running microframework. Could it be as easy as
> bottle: download and run ;-)
>
> I will try it and soon I reply....

Well, pip /should/ have worked. If you want to skip my suggestion,
just try figuring out if the version of Python pip is running is the
same as your base python binary. Or maybe ask in the pip/virtualenv
list[1]?

One thing I forgot in my "microtutorial" is that you also need to do
the same for Werkzeug and Jinja2, which are Flask dependencies. Both
are available from Github too in Armin's profile[2].

-- Jonas

[1] http://groups.google.com/group/python-virtualenv?hl=en
[2] http://github.com/mitsuhiko

Re: [flask] Installation problems via pip

From:
kevin beckford
Date:
2010-10-19 @ 06:16
> You can save yourself all this trouble by learning how modules are
> loaded in Python (which is really simple[1]),


Disagree.  Virtualenv and buildout exist for this very reason. Python
modules and python  sys.path calculations are counter intuitive.

http://www.python.org/dev/peps/pep-0370 is an option also.

Clarification:  I disagree with the "( which is really simple[1])"
only.  Module loading in python is an essential thing to know to avoid
constant and aggravating surprises.

Re: [flask] Installation problems via pip

From:
Jonas Galvez
Date:
2010-10-19 @ 14:03
On Tue, Oct 19, 2010 at 4:16 AM, kevin beckford <chiggsy@lazyweb.ca> wrote:
> Disagree.  Virtualenv and buildout exist for this very reason. Python
> modules and python  sys.path calculations are counter intuitive.

I think pip and buildout promote a certain degree of ignorance over
how Python loads modules, especially for novice users, that's why I
pointed him to the Python docs.

I think pip is great, and I use it on my laptop for quickly installing
libraries for system-wide use. But for specific projects, I have found
that "freezing" the libraries your project depends on in the project's
trunk itself really pays off in the long run. Problems this approach
avoids:

- You've just pulled your project on a new server box and start
installing all libraries via pip. Imagine if a library got an update
that introduced some nasty little bug that affects a library function
you're using. (Happened to me).

- Your boss says you need to process 250.000 files in 3 days. You boot
up 40+ EC2 instances to do the job. You're forced to write a script to
install everything on each instance. (Happened to me). If I had kept
libraries frozen, most of them would be instantly available after git
cloning them into the machines. And only a few would require local
compilation. (Of course the best approach here is have a custom server
image with all dependencies pre-loaded, but that's yet another
motivation for keeping all your dependencies in a single, manageable
location).

-- Jonas

Re: [flask] Installation problems via pip

From:
kevin beckford
Date:
2010-10-19 @ 16:30
> I think pip and buildout promote a certain degree of ignorance over
> how Python loads modules, especially for novice users, that's why I
> pointed him to the Python docs.

Pip is more a tool like easy_install.  The path from the python
distutils docs, all the way to installing via pip ( the only installer
that has uninstall  at this point ) makes little sense for a culture
professing 'There should only be one way to do it.'

Buildout and virtualenv both provide the freezing you speak of, albeit
in different ways.
requirements.txt for virtualenv and
http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used
for buildout.

Personally I prefer Buildout for applications since it has more
functionality,  but many people use virtualenv because of it's
simplicity.  I use virtualenv myself if I want to quickly test
something, although pip's support for uninstall is reducing even that
usage.

Summary:
pip <--> easy_install
virtualenv <--> buildout

Re: [flask] Installation problems via pip

From:
Jonas Galvez
Date:
2010-10-19 @ 16:42
On Tue, Oct 19, 2010 at 2:30 PM, kevin beckford <chiggsy@lazyweb.ca> wrote:
> Pip is more a tool like easy_install.

Yeah, I totally meant easy_install when I wrote buildout. I actually
haven't looked at buildout yet. I need to take a day off and revisit
this whole thread. Lots of good insight :)

-- Jonas

Re: [flask] Installation problems via pip

From:
Dag Odenhall
Date:
2010-10-19 @ 14:24
On Tue, 2010-10-19 at 12:03 -0200, Jonas Galvez wrote:
> On Tue, Oct 19, 2010 at 4:16 AM, kevin beckford <chiggsy@lazyweb.ca> wrote:
> > Disagree.  Virtualenv and buildout exist for this very reason. Python
> > modules and python  sys.path calculations are counter intuitive.
> 
> I think pip and buildout promote a certain degree of ignorance over
> how Python loads modules, especially for novice users, that's why I
> pointed him to the Python docs.
> 
> I think pip is great, and I use it on my laptop for quickly installing
> libraries for system-wide use. But for specific projects, I have found
> that "freezing" the libraries your project depends on in the project's
> trunk itself really pays off in the long run. Problems this approach
> avoids:
> 
> - You've just pulled your project on a new server box and start
> installing all libraries via pip. Imagine if a library got an update
> that introduced some nasty little bug that affects a library function
> you're using. (Happened to me).
> 
> - Your boss says you need to process 250.000 files in 3 days. You boot
> up 40+ EC2 instances to do the job. You're forced to write a script to
> install everything on each instance. (Happened to me). If I had kept
> libraries frozen, most of them would be instantly available after git
> cloning them into the machines. And only a few would require local
> compilation. (Of course the best approach here is have a custom server
> image with all dependencies pre-loaded, but that's yet another
> motivation for keeping all your dependencies in a single, manageable
> location).
> 
> -- Jonas

Or use distribute and write a proper setup.py. You can restrict
dependencies to specific versions or ranges.

http://flask.pocoo.org/docs/patterns/distribute/

An alternative is to install dependencies in a virtualenv and use pip to
freeze:

pip freeze -l >requirements.txt

Install:

pip install -r requirements.txt

though I prefer to have a proper setup.py; helps you run tests, build
source packages, documentation etc etc.

Re: [flask] Installation problems via pip

From:
Jonas Galvez
Date:
2010-10-19 @ 14:42
On Tue, Oct 19, 2010 at 12:24 PM, Dag Odenhall <dag.odenhall@gmail.com> wrote:
> An alternative is to install dependencies in a virtualenv and use pip to
> freeze:
>
> pip freeze -l >requirements.txt
>
> Install:
>
> pip install -r requirements.txt
>
> though I prefer to have a proper setup.py; helps you run tests, build
> source packages, documentation etc etc.

Ha, didn't know about "pip freeze", that sure is handy. But it still
needs to download the packages when you run pip install, right? That's
just the bit that still makes me a little uneasy. I can't have a
project depend on a package file being available in some remote
location when I need to install and get it running "now". Depending on
a single remote location is already enough trouble :)

virtualenv is great, and in a way, my "adhoc way" of handling packages
mimics what virtualenv does, with the exception that I don't feel the
need to cache the Python binary itself under my project. I just
basically always assume I have at least Python 2.5 on whatever server
box I'm running (and that, thankfully, is usually the case).

Thanks for tip!

-- Jonas

Re: [flask] Installation problems via pip

From:
Dag Odenhall
Date:
2010-10-19 @ 16:26
On Tue, 2010-10-19 at 12:42 -0200, Jonas Galvez wrote:
> 
> Ha, didn't know about "pip freeze", that sure is handy. But it still
> needs to download the packages when you run pip install, right? That's
> just the bit that still makes me a little uneasy. I can't have a
> project depend on a package file being available in some remote
> location when I need to install and get it running "now". Depending on
> a single remote location is already enough trouble :) 

You can create and install bundles with pip:

pip freeze -l >requirements.txt
pip bundle -r requirements.txt packages.pybundle

pip install packages.pybundle

http://pip.openplans.org/#bundles

It's probably possible to use distribute in similar ways. The issue with
your method is that it's not portable* or easily upgradeable. The
benefit I suppose is that you can modify dependencies easily as you
already effectively have forks of them all. Though that might keep you
from contributing your changes upstream. ;)

*Depending on what you meant by freezing. If you're bundling installs
it's not always portable to other machines different from your work
station. If you're bundling the sources it'll be fine but then you'll
need to recursively run their install scripts anyway and might as well
make use of existing solutions to this problem.

Re: [flask] Installation problems via pip

From:
kevin beckford
Date:
2010-10-19 @ 17:06
>> project depend on a package file being available in some remote
>> location when I need to install and get it running "now". Depending on
>> a single remote location is already enough trouble :)

For completeness, you can also use buildout's cache to avoid going
over the network if you find it undependable.

http://pypi.python.org/pypi/zc.buildout#using-the-cache-purely-as-a-fall-back

Two excellent options, buildout and virtualenv.

Re: [flask] Installation problems via pip

From:
Jonas Galvez
Date:
2010-10-19 @ 16:51
On Tue, Oct 19, 2010 at 2:26 PM, Dag Odenhall <dag.odenhall@gmail.com> wrote:
> It's probably possible to use distribute in similar ways. The issue with
> your method is that it's not portable* or easily upgradeable. The
> benefit I suppose is that you can modify dependencies easily as you
> already effectively have forks of them all. Though that might keep you
> from contributing your changes upstream. ;)
>
> *Depending on what you meant by freezing. If you're bundling installs
> it's not always portable to other machines different from your work
> station. If you're bundling the sources it'll be fine but then you'll
> need to recursively run their install scripts anyway and might as well
> make use of existing solutions to this problem.

Freezing the sources under $project/lib/ is portable if no C
extensions are required to compile.

That's the only problem I still need to solve in my adhoc approach.
Whenever I need some library that has C extensions (like lxml), I just
keep the entire lxml source code under lib (i.e., $project/lib/lxml)
and compile manually.

What I really want is a Fabric extension that has tasks like "addlib"
and "compilelibs". "fab addlib" would take a Bitbucket or Github link
(or pip identifier) and just do whatever's needed to get you a clean
directory under in "./lib", and "fab compilelibs" would get all the C
extensions compiled locally under your project. I'll try and implement
that soon.

Let me just emphasize that I'm not against pip. I'm just one of those
people who really need to know and control what's going on under the
hood. So I guess part of me just wants to go ahead and write it so I
can learn more in the process.

-- Jonas

Re: [flask] Installation problems via pip

From:
kevin beckford
Date:
2010-10-19 @ 18:13
> What I really want is a Fabric extension that has tasks like "addlib"
> and "compilelibs". "fab addlib" would take a Bitbucket or Github link
> (or pip identifier) and just do whatever's needed to get you a clean
> directory under in "./lib", and "fab compilelibs" would get all the C
> extensions compiled locally under your project. I'll try and implement
> that soon.

I feel your pain.  You want python to work like a standard unix
program, where you say "Look here first for your data."  In Perl it
was as simple as:

http://perldoc.perl.org/lib.html

#!/usr/bin/env perl

# The wheel is faulty! I have re-invented it here!
# Look upon my works ye mighty, and despair!

use lib 'Wheel2';


I found the whole virtualenv concept somewhat odd, since one
ordinarily does not virtualize interpreters but entire systems.  I've
learned to accept this about python.  I still feel your pain though.

Re: [flask] Installation problems via pip

From:
danjac354@gmail.com
Date:
2010-10-19 @ 19:09
> I feel your pain.  You want python to work like a standard unix
> program, where you say "Look here first for your data."  In Perl it
> was as simple as:
>
> http://perldoc.perl.org/lib.html
>
> #!/usr/bin/env perl
>
> # The wheel is faulty! I have re-invented it here!
> # Look upon my works ye mighty, and despair!
>
> use lib 'Wheel2';

Whenever there's a new feature in Python or Ruby you find Perl has had
it for years....

Re: [flask] Installation problems via pip

From:
Kenneth Reitz
Date:
2010-10-19 @ 14:43
Jonas,

Check out virtualenvwrapper. It makes making/using virtualenv's a breeze.

http://www.doughellmann.com/projects/virtualenvwrapper/

Kenneth Reitz
http://kennethreitz.com/contact-me



On Tue, Oct 19, 2010 at 10:42 AM, Jonas Galvez <jonasgalvez@gmail.com>wrote:

> On Tue, Oct 19, 2010 at 12:24 PM, Dag Odenhall <dag.odenhall@gmail.com>
> wrote:
> > An alternative is to install dependencies in a virtualenv and use pip to
> > freeze:
> >
> > pip freeze -l >requirements.txt
> >
> > Install:
> >
> > pip install -r requirements.txt
> >
> > though I prefer to have a proper setup.py; helps you run tests, build
> > source packages, documentation etc etc.
>
> Ha, didn't know about "pip freeze", that sure is handy. But it still
> needs to download the packages when you run pip install, right? That's
> just the bit that still makes me a little uneasy. I can't have a
> project depend on a package file being available in some remote
> location when I need to install and get it running "now". Depending on
> a single remote location is already enough trouble :)
>
> virtualenv is great, and in a way, my "adhoc way" of handling packages
> mimics what virtualenv does, with the exception that I don't feel the
> need to cache the Python binary itself under my project. I just
> basically always assume I have at least Python 2.5 on whatever server
> box I'm running (and that, thankfully, is usually the case).
>
> Thanks for tip!
>
> -- Jonas
>

Re: [flask] Installation problems via pip

From:
DasIch
Date:
2010-10-19 @ 14:22
You can just keep a requirements.txt file which specifies the version
for each dependency you install with pip.

If you need to make multiple deployments you could create a virtual
environment, install everything into that and just copy it.

Re: [flask] Installation problems via pip

From:
Dag Odenhall
Date:
2010-10-19 @ 16:33
On Tue, 2010-10-19 at 16:22 +0200, DasIch wrote:
> You can just keep a requirements.txt file which specifies the version
> for each dependency you install with pip.
> 
> If you need to make multiple deployments you could create a virtual
> environment, install everything into that and just copy it.

Doesn't sound terribly portable, especially regarding C extensions but
possibly other issues as well.