Hi,
I have write a config module to load configuration from mongoDB
database, you can see the code source on github.
https://github.com/wysman/mongrel2/tree/config-from-mongodb
This module allow you to retrieve configuration from a single mongoDB
server or a shard server.
I use the configuration path variable to pass a custom URI to the
database, as you can see below.
./bin/mongrel2 "srv:127.0.0.1:mongrel2"
971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so
./bin/mongrel2 "shard:rs_a:127.0.0.1@27017;127.0.0.1@27018:mongrel2"
971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so
I think the code is near to be ready for a pull request, but i have a
last issue.
When i build mongrel2 with :
- "make clean all", all works fine.
- "make clean dev", all compile, but during the dlopen at execution, i
have issue with unknown symbol.
[INFO] (src/mongrel2.c:379) Using configuration module
tools/config_modules/mongodb.so to load configs.
./bin/mongrel2: symbol lookup error:
tools/config_modules/mongodb.so: undefined symbol: dbg_get_log
I need some help, because i am not an expert about dynamic module loading in C.
Some comment and code review will be appreciate.
Thanks
William
2011/12/28 William MARTIN <wysman@gmail.com>: > Hi, > > I have write a config module to load configuration from mongoDB > database, you can see the code source on github. > https://github.com/wysman/mongrel2/tree/config-from-mongodb > > This module allow you to retrieve configuration from a single mongoDB > server or a shard server. > I use the configuration path variable to pass a custom URI to the > database, as you can see below. > > ./bin/mongrel2 "srv:127.0.0.1:mongrel2" > 971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so > ./bin/mongrel2 "shard:rs_a:127.0.0.1@27017;127.0.0.1@27018:mongrel2" > 971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so > That's awesome!! I will try to take a look at this more closely. I will also have to install mongo, as I never used it. Prepare to help me on some issue along the way! =) > I think the code is near to be ready for a pull request, but i have a > last issue. > > When i build mongrel2 with : > > - "make clean all", all works fine. > - "make clean dev", all compile, but during the dlopen at execution, i > have issue with unknown symbol. > > [INFO] (src/mongrel2.c:379) Using configuration module > tools/config_modules/mongodb.so to load configs. > ./bin/mongrel2: symbol lookup error: > tools/config_modules/mongodb.so: undefined symbol: dbg_get_log > > I need some help, because i am not an expert about dynamic module loading in C. > Some comment and code review will be appreciate. > That's strange. The debug macro works on null.c config module and after compiling I took a (quick) look at both binaries: $ strings tools/config_modules/null.so | grep get_log dbg_get_log $ strings tools/config_modules/mongodb.so | grep get_log dbg_get_log although mongodb.so does not work. I will try to take a look later. -- Dalton Barreto http://daltonmatos.com
Hi, 1) Install mongo You can download mongoDB here (http://www.mongodb.org/downloads 2) Run the server mkdir -p /tmp/mongodata ./bin/mongod -dbpath /tmp/mongodata 3) Load a config into the I have push a simple script to create database content. https://github.com/wysman/mongrel2/commit/31ada1182d5ef46fe3908b31bf6a2ec92bdcbff3 We can't directly use the shell of mongo to create it because this shell use Javascript, and all number are converted to double. 3) Run mongrel2 ./bin/mongrel2 "srv:127.0.0.1:mongrel2" 971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so Tell me if you have issues. William On Thu, Dec 29, 2011 at 6:26 PM, Dalton Barreto <daltonmatos@gmail.com> wrote: > 2011/12/28 William MARTIN <wysman@gmail.com>: >> Hi, >> >> I have write a config module to load configuration from mongoDB >> database, you can see the code source on github. >> https://github.com/wysman/mongrel2/tree/config-from-mongodb >> >> This module allow you to retrieve configuration from a single mongoDB >> server or a shard server. >> I use the configuration path variable to pass a custom URI to the >> database, as you can see below. >> >> ./bin/mongrel2 "srv:127.0.0.1:mongrel2" >> 971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so >> ./bin/mongrel2 "shard:rs_a:127.0.0.1@27017;127.0.0.1@27018:mongrel2" >> 971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so >> > > That's awesome!! I will try to take a look at this more closely. I > will also have to > install mongo, as I never used it. Prepare to help me on some issue > along the way! =) > >> I think the code is near to be ready for a pull request, but i have a >> last issue. >> >> When i build mongrel2 with : >> >> - "make clean all", all works fine. >> - "make clean dev", all compile, but during the dlopen at execution, i >> have issue with unknown symbol. >> >> [INFO] (src/mongrel2.c:379) Using configuration module >> tools/config_modules/mongodb.so to load configs. >> ./bin/mongrel2: symbol lookup error: >> tools/config_modules/mongodb.so: undefined symbol: dbg_get_log >> >> I need some help, because i am not an expert about dynamic module loading in C. >> Some comment and code review will be appreciate. >> > > That's strange. The debug macro works on null.c config module and > after compiling I took a (quick) look at both binaries: > > $ strings tools/config_modules/null.so | grep get_log > dbg_get_log > > $ strings tools/config_modules/mongodb.so | grep get_log > dbg_get_log > > although mongodb.so does not work. I will try to take a look later. > > > -- > Dalton Barreto > http://daltonmatos.com -- --------------------------------------------------------- William MARTIN wysman @NoSpAm@ gmail @DoT@ com
2011/12/30 William MARTIN <wysman@gmail.com>: > Hi, > > 1) Install mongo > You can download mongoDB here > (http://www.mongodb.org/downloads > > 2) Run the server > mkdir -p /tmp/mongodata > ./bin/mongod -dbpath /tmp/mongodata > > 3) Load a config into the > I have push a simple script to create database content. > https://github.com/wysman/mongrel2/commit/31ada1182d5ef46fe3908b31bf6a2ec92bdcbff3 > We can't directly use the shell of mongo to create it because this > shell use Javascript, and all number are converted to double. > > 3) Run mongrel2 > ./bin/mongrel2 "srv:127.0.0.1:mongrel2" > 971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so > > Tell me if you have issues. > William Hello William, I tried your mongodb config module but got an error: http://codepad.org/vD57SHUd But there is something very important to consider, before assuming the error comes from the code: * I reconstructed the mongodb database *by hand*, using the python shell (pymongo). That's because I don't have any php tools installed and this would give me an extra work to install. Considering this is my very first contact with mongo, probably I did something wrong. =) Could you upload a db-dump of your mongo db somewhere? Maybe this way we ensure that we have the same database. And here is a nano-patch to your code, is just a typo: diff --git a/tools/config_modules/mongodb.c b/tools/config_modules/mongodb.c index 860957a..0484272 100644 --- a/tools/config_modules/mongodb.c +++ b/tools/config_modules/mongodb.c @@ -45,7 +45,7 @@ static bstring dbname; #define MONGODB_IP_SEPARATOR ';' #define MONGODB_TOKEN_SEPARATOR ':' #define MONGODB_TOKEN_SHARD "shard" -#define MONGODB_TOKEN_SERVER "srtv" +#define MONGODB_TOKEN_SERVER "srv" static int config_init_index(void) { Thanks! -- Dalton Barreto http://daltonmatos.com
Hi, Happy new year ! I have attached my mongrel2 database. You can restore it with this line. ./bin/mongorestore --db mongrel2 <your path>/mongrel2/ Can you export your database with this line, so i will test with it on my laptop. ./bin/mongodump --db mongrel2 --out foo Thanks William On Sat, Dec 31, 2011 at 9:13 PM, Dalton Barreto <daltonmatos@gmail.com> wrote: > 2011/12/30 William MARTIN <wysman@gmail.com>: >> Hi, >> >> 1) Install mongo >> You can download mongoDB here >> (http://www.mongodb.org/downloads >> >> 2) Run the server >> mkdir -p /tmp/mongodata >> ./bin/mongod -dbpath /tmp/mongodata >> >> 3) Load a config into the >> I have push a simple script to create database content. >> https://github.com/wysman/mongrel2/commit/31ada1182d5ef46fe3908b31bf6a2ec92bdcbff3 >> We can't directly use the shell of mongo to create it because this >> shell use Javascript, and all number are converted to double. >> >> 3) Run mongrel2 >> ./bin/mongrel2 "srv:127.0.0.1:mongrel2" >> 971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so >> >> Tell me if you have issues. >> William > > Hello William, > > I tried your mongodb config module but got an error: http://codepad.org/vD57SHUd > > But there is something very important to consider, before assuming > the error comes from the code: > > * I reconstructed the mongodb database *by hand*, using the python > shell (pymongo). That's because I don't have any php tools installed > and this would give me an extra work to install. Considering this is > my very first contact with mongo, probably I did something wrong. =) > > Could you upload a db-dump of your mongo db somewhere? Maybe this way > we ensure that we have the same database. > > And here is a nano-patch to your code, is just a typo: > > diff --git a/tools/config_modules/mongodb.c b/tools/config_modules/mongodb.c > index 860957a..0484272 100644 > --- a/tools/config_modules/mongodb.c > +++ b/tools/config_modules/mongodb.c > @@ -45,7 +45,7 @@ static bstring dbname; > #define MONGODB_IP_SEPARATOR ';' > #define MONGODB_TOKEN_SEPARATOR ':' > #define MONGODB_TOKEN_SHARD "shard" > -#define MONGODB_TOKEN_SERVER "srtv" > +#define MONGODB_TOKEN_SERVER "srv" > > static int config_init_index(void) > { > > > Thanks! > > > -- > Dalton Barreto > http://daltonmatos.com -- --------------------------------------------------------- William MARTIN wysman @NoSpAm@ gmail @DoT@ com
I think your issue will be fixed with this commit https://github.com/wysman/mongrel2/commit/b7dd4035efbfba26e8d122cd8808d8a4e21a8086 William On Sun, Jan 1, 2012 at 3:18 PM, William MARTIN <wysman@gmail.com> wrote: > Hi, > > Happy new year ! > > I have attached my mongrel2 database. > You can restore it with this line. > ./bin/mongorestore --db mongrel2 <your path>/mongrel2/ > > Can you export your database with this line, so i will test with it on > my laptop. > ./bin/mongodump --db mongrel2 --out foo > > Thanks > William > > On Sat, Dec 31, 2011 at 9:13 PM, Dalton Barreto <daltonmatos@gmail.com> wrote: >> 2011/12/30 William MARTIN <wysman@gmail.com>: >>> Hi, >>> >>> 1) Install mongo >>> You can download mongoDB here >>> (http://www.mongodb.org/downloads >>> >>> 2) Run the server >>> mkdir -p /tmp/mongodata >>> ./bin/mongod -dbpath /tmp/mongodata >>> >>> 3) Load a config into the >>> I have push a simple script to create database content. >>> https://github.com/wysman/mongrel2/commit/31ada1182d5ef46fe3908b31bf6a2ec92bdcbff3 >>> We can't directly use the shell of mongo to create it because this >>> shell use Javascript, and all number are converted to double. >>> >>> 3) Run mongrel2 >>> ./bin/mongrel2 "srv:127.0.0.1:mongrel2" >>> 971e0536-a92d-41e2-bea3-9af54bfd6fd9 tools/config_modules/mongodb.so >>> >>> Tell me if you have issues. >>> William >> >> Hello William, >> >> I tried your mongodb config module but got an error: http://codepad.org/vD57SHUd >> >> But there is something very important to consider, before assuming >> the error comes from the code: >> >> * I reconstructed the mongodb database *by hand*, using the python >> shell (pymongo). That's because I don't have any php tools installed >> and this would give me an extra work to install. Considering this is >> my very first contact with mongo, probably I did something wrong. =) >> >> Could you upload a db-dump of your mongo db somewhere? Maybe this way >> we ensure that we have the same database. >> >> And here is a nano-patch to your code, is just a typo: >> >> diff --git a/tools/config_modules/mongodb.c b/tools/config_modules/mongodb.c >> index 860957a..0484272 100644 >> --- a/tools/config_modules/mongodb.c >> +++ b/tools/config_modules/mongodb.c >> @@ -45,7 +45,7 @@ static bstring dbname; >> #define MONGODB_IP_SEPARATOR ';' >> #define MONGODB_TOKEN_SEPARATOR ':' >> #define MONGODB_TOKEN_SHARD "shard" >> -#define MONGODB_TOKEN_SERVER "srtv" >> +#define MONGODB_TOKEN_SERVER "srv" >> >> static int config_init_index(void) >> { >> >> >> Thanks! >> >> >> -- >> Dalton Barreto >> http://daltonmatos.com > > > > -- > --------------------------------------------------------- > William MARTIN > wysman @NoSpAm@ gmail @DoT@ com -- --------------------------------------------------------- William MARTIN wysman @NoSpAm@ gmail @DoT@ com
2012/1/1 William MARTIN <wysman@gmail.com>: > I think your issue will be fixed with this commit > https://github.com/wysman/mongrel2/commit/b7dd4035efbfba26e8d122cd8808d8a4e21a8086 > Indeed! It worked!! I was able to serve a simple django app using wsgid as the backend. Maybe it's a good idea to put a db dump somewhere, so people will be able to test your module more easily. Here is a dump of my db and I'm pretty sure that is was really broken. =) I created a fresh mongodb using your dbdump and it worked! Thanks for your module and I will probably use it as a starting point (mainly to see how the tnetstring stuff is handled) to write my own. -- Dalton Barreto http://daltonmatos.com