librelist archives

« back to archive

CTCoreMessage rfc822

CTCoreMessage rfc822

From:
Olivier Palliere
Date:
2012-09-20 @ 15:04
Hi Guys,

Still working on my mail client, I'm stumbling onto a weird problem. I 
have to scan an account withe about 3800 messages.

Everything works fine in the beginning and for mailboxes that have a 
limited number of messages. But when I'm on that big one, after a couple 
of messages, it start acting wrong.

The - rfc822 method always returns NULL, and thus, the string I'd like to 
save somewhere is also null.

Any idea where I should look?

I'm happy to give some code, but it won't be pretty ;-)

Many thanks,
Olivier./.

Re: [mailcore] CTCoreMessage rfc822

From:
Matt Ronge
Date:
2012-09-20 @ 15:08
Hi Olivier,

First try turning on MailCoreEnableLogging() and if you could please send 
a copy of the output from the console. That will give us an idea of if 
data is coming back properly from the server.

BTW: The log will contain passwords and other login information, so make 
sure to scrub that out
--
Matt Ronge

iOS Developer & Consultant
Central Atomics Inc.
http://www.centralatomics.com






On Sep 20, 2012, at 10:04 AM, Olivier Palliere <olivier@molowa.com> wrote:

> Hi Guys,
> 
> Still working on my mail client, I'm stumbling onto a weird problem. I 
have to scan an account withe about 3800 messages.
> 
> Everything works fine in the beginning and for mailboxes that have a 
limited number of messages. But when I'm on that big one, after a couple 
of messages, it start acting wrong.
> 
> The - rfc822 method always returns NULL, and thus, the string I'd like 
to save somewhere is also null.
> 
> Any idea where I should look?
> 
> I'm happy to give some code, but it won't be pretty ;-)
> 
> Many thanks,
> Olivier./.
> 

Re: [mailcore] CTCoreMessage rfc822

From:
Olivier Palliere
Date:
2012-09-20 @ 15:12
Hi Matt,

Will do. In the meantime, the lastError property on my CTCoreMessage is 
returning: Parse Error


Olivier./.

On Sep 20, 2012, at 5:08 PM, Matt Ronge <mronge@mronge.com> wrote:

> Hi Olivier,
> 
> First try turning on MailCoreEnableLogging() and if you could please 
send a copy of the output from the console. That will give us an idea of 
if data is coming back properly from the server.
> 
> BTW: The log will contain passwords and other login information, so make
sure to scrub that out
> --
> Matt Ronge
> 
> iOS Developer & Consultant
> Central Atomics Inc.
> http://www.centralatomics.com
> 
> 
> 
> 
> 
> 
> On Sep 20, 2012, at 10:04 AM, Olivier Palliere <olivier@molowa.com> wrote:
> 
>> Hi Guys,
>> 
>> Still working on my mail client, I'm stumbling onto a weird problem. I 
have to scan an account withe about 3800 messages.
>> 
>> Everything works fine in the beginning and for mailboxes that have a 
limited number of messages. But when I'm on that big one, after a couple 
of messages, it start acting wrong.
>> 
>> The - rfc822 method always returns NULL, and thus, the string I'd like 
to save somewhere is also null.
>> 
>> Any idea where I should look?
>> 
>> I'm happy to give some code, but it won't be pretty ;-)
>> 
>> Many thanks,
>> Olivier./.
>> 
> 

Re: [mailcore] CTCoreMessage rfc822

From:
Olivier Palliere
Date:
2012-09-20 @ 15:32
Hi Matt and all,

I'll send the logs when XCode gives me the focus back (kinda colorwheel of
death right now ;-))

But form what I see, is it possible that whle replying to a FETCH with a 
very long video, other requests coming for a FETCH might be somehow queued
and not giving back a value.

Form what I saw, I have a mail where a wmv is attached, and I keep getting 

>>>> send

941 FETCH 937 RFC822

>>>>> end send

while I get some 

<<<<<< read

kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
elrgowe7gow837gopw397rghow3iurfgow]wigf
;wurfghw3974ghpw374ghpw

...

lots of data

...

<<<<<< end read

then later again a 


<<<<<< read

kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
elrgowe7gow837gopw397rghow3iurfgow]wigf
;wurfghw3974ghpw374ghpw

...

lots of data

...

<<<<<< end read


that could be the continuation of a previous FETCH, hence blocking me?

Many thanks in any case,
Olivier./.

On Sep 20, 2012, at 5:12 PM, Olivier Palliere <olivier@molowa.com> wrote:

> Hi Matt,
> 
> Will do. In the meantime, the lastError property on my CTCoreMessage is 
returning: Parse Error
> 
> 
> Olivier./.
> 
> On Sep 20, 2012, at 5:08 PM, Matt Ronge <mronge@mronge.com> wrote:
> 
>> Hi Olivier,
>> 
>> First try turning on MailCoreEnableLogging() and if you could please 
send a copy of the output from the console. That will give us an idea of 
if data is coming back properly from the server.
>> 
>> BTW: The log will contain passwords and other login information, so 
make sure to scrub that out
>> --
>> Matt Ronge
>> 
>> iOS Developer & Consultant
>> Central Atomics Inc.
>> http://www.centralatomics.com
>> 
>> 
>> 
>> 
>> 
>> 
>> On Sep 20, 2012, at 10:04 AM, Olivier Palliere <olivier@molowa.com> wrote:
>> 
>>> Hi Guys,
>>> 
>>> Still working on my mail client, I'm stumbling onto a weird problem. I
have to scan an account withe about 3800 messages.
>>> 
>>> Everything works fine in the beginning and for mailboxes that have a 
limited number of messages. But when I'm on that big one, after a couple 
of messages, it start acting wrong.
>>> 
>>> The - rfc822 method always returns NULL, and thus, the string I'd like
to save somewhere is also null.
>>> 
>>> Any idea where I should look?
>>> 
>>> I'm happy to give some code, but it won't be pretty ;-)
>>> 
>>> Many thanks,
>>> Olivier./.
>>> 
>> 
> 

Re: [mailcore] CTCoreMessage rfc822

From:
Olivier Palliere
Date:
2012-09-21 @ 08:13
Hi Guys,

Further testing. This is for the initial scan of a mailbox, that has 3800 
messages. Right now, I'm getting the array of all messages using:

  	NSArray *messageSet = [aFolder messagesFromSequenceNumber:1 to:0 
withFetchAttributes:CTFetchAttrEnvelope ];


Then looping through the array to get the infos I need for the messages:

       for (CTCoreMessage *msg in messageSet)
        {
            if ([self createMessage:msg 
inManagedObjectContext:localContext inCDMailbox:newBox])
            {
                NSLog(@"Message created (%ld)", currentStep);
                
            } else
            {
                NSLog(@"Error creating message %ld", currentStep);
                
            }

In the createMessage method, I assign all the fields to my CoreData entity
using the CTCoreMessage:

  	  Messages *newMail = [Messages MR_createInContext:localContext]; // 
Using MagicalRecord now
    
	  [msg fetchBodyStructure];
  	  
	   newMail.uid = [NSNumber numberWithInteger:msg.uid];
 	   newMail.messageID = msg.messageId;
           newMail.senderName = msg.sender.email;
	....
	and so on

And at some point, I also ask the server to give me the rfc822 content for
the message, to save it somewhere on disk

	NSString *rfc822 = msg.rfc822;
	NSError *writeError;
	if (![rfc822 writeToFile:localPath atomically:YES 
encoding:NSUTF8StringEncoding error:&writeError])
	{
		NSLog(@"error: %@", writeError.localizedFailureReason);
		NSLog(@"rfx822 : %@ NOTNOTNOTNOT saved for %@", rfc822, newMail.mailbox.path );
		NSLog(@"lastError: %@", [msg.lastError localizedDescription]);
	}

Well, that suddenly hangs after about 1500 messages, and the rfc822 is 
only written for the couple first messages.

Today I tried to comment out the rfc822 part in my code, and boom, the 
scan is working fine. It looks like asking the rfc822 is quite costly on 
the server, as it just can't cope and just stop answering. So I'm thinking
I'll have to stick with the request of the body structure and get the 
rfc822 later in the code, or using a dedicated thread that checks the 
answer is here before I try the next messages.

Does that make sense?

Many thanks for the help,
Olivier./.


On Sep 20, 2012, at 5:32 PM, Olivier Palliere <olivier@molowa.com> wrote:

> Hi Matt and all,
> 
> I'll send the logs when XCode gives me the focus back (kinda colorwheel 
of death right now ;-))
> 
> But form what I see, is it possible that whle replying to a FETCH with a
very long video, other requests coming for a FETCH might be somehow queued
and not giving back a value.
> 
> Form what I saw, I have a mail where a wmv is attached, and I keep getting 
> 
>>>>> send
> 
> 941 FETCH 937 RFC822
> 
>>>>>> end send
> 
> while I get some 
> 
> <<<<<< read
> 
> kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
> elrgowe7gow837gopw397rghow3iurfgow]wigf
> ;wurfghw3974ghpw374ghpw
> 
> ...
> 
> lots of data
> 
> ...
> 
> <<<<<< end read
> 
> then later again a 
> 
> 
> <<<<<< read
> 
> kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
> elrgowe7gow837gopw397rghow3iurfgow]wigf
> ;wurfghw3974ghpw374ghpw
> 
> ...
> 
> lots of data
> 
> ...
> 
> <<<<<< end read
> 
> 
> that could be the continuation of a previous FETCH, hence blocking me?
> 
> Many thanks in any case,
> Olivier./.
> 
> On Sep 20, 2012, at 5:12 PM, Olivier Palliere <olivier@molowa.com> wrote:
> 
>> Hi Matt,
>> 
>> Will do. In the meantime, the lastError property on my CTCoreMessage is
returning: Parse Error
>> 
>> 
>> Olivier./.
>> 
>> On Sep 20, 2012, at 5:08 PM, Matt Ronge <mronge@mronge.com> wrote:
>> 
>>> Hi Olivier,
>>> 
>>> First try turning on MailCoreEnableLogging() and if you could please 
send a copy of the output from the console. That will give us an idea of 
if data is coming back properly from the server.
>>> 
>>> BTW: The log will contain passwords and other login information, so 
make sure to scrub that out
>>> --
>>> Matt Ronge
>>> 
>>> iOS Developer & Consultant
>>> Central Atomics Inc.
>>> http://www.centralatomics.com
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Sep 20, 2012, at 10:04 AM, Olivier Palliere <olivier@molowa.com> wrote:
>>> 
>>>> Hi Guys,
>>>> 
>>>> Still working on my mail client, I'm stumbling onto a weird problem. 
I have to scan an account withe about 3800 messages.
>>>> 
>>>> Everything works fine in the beginning and for mailboxes that have a 
limited number of messages. But when I'm on that big one, after a couple 
of messages, it start acting wrong.
>>>> 
>>>> The - rfc822 method always returns NULL, and thus, the string I'd 
like to save somewhere is also null.
>>>> 
>>>> Any idea where I should look?
>>>> 
>>>> I'm happy to give some code, but it won't be pretty ;-)
>>>> 
>>>> Many thanks,
>>>> Olivier./.
>>>> 
>>> 
>> 
> 

Re: [mailcore] CTCoreMessage rfc822

From:
Jonathan Willing
Date:
2012-09-21 @ 13:35
Hi,

You really need to run this through Instruments' profiler tool to see 
exactly what is holding it up. Making guesses about optimization is never 
a good idea.

Jonathan

On Sep 21, 2012, at 3:13 AM, Olivier Palliere <olivier@molowa.com> wrote:

> Hi Guys,
> 
> Further testing. This is for the initial scan of a mailbox, that has 
3800 messages. Right now, I'm getting the array of all messages using:
> 
>  	NSArray *messageSet = [aFolder messagesFromSequenceNumber:1 to:0 
withFetchAttributes:CTFetchAttrEnvelope ];
> 
> 
> Then looping through the array to get the infos I need for the messages:
> 
>       for (CTCoreMessage *msg in messageSet)
>        {
>            if ([self createMessage:msg 
inManagedObjectContext:localContext inCDMailbox:newBox])
>            {
>                NSLog(@"Message created (%ld)", currentStep);
> 
>            } else
>            {
>                NSLog(@"Error creating message %ld", currentStep);
> 
>            }
> 
> In the createMessage method, I assign all the fields to my CoreData 
entity using the CTCoreMessage:
> 
>  	  Messages *newMail = [Messages MR_createInContext:localContext]; // 
Using MagicalRecord now
> 
> 	  [msg fetchBodyStructure];
>  	  
> 	   newMail.uid = [NSNumber numberWithInteger:msg.uid];
> 	   newMail.messageID = msg.messageId;
>           newMail.senderName = msg.sender.email;
> 	....
> 	and so on
> 
> And at some point, I also ask the server to give me the rfc822 content 
for the message, to save it somewhere on disk
> 
> 	NSString *rfc822 = msg.rfc822;
> 	NSError *writeError;
> 	if (![rfc822 writeToFile:localPath atomically:YES 
encoding:NSUTF8StringEncoding error:&writeError])
> 	{
> 		NSLog(@"error: %@", writeError.localizedFailureReason);
> 		NSLog(@"rfx822 : %@ NOTNOTNOTNOT saved for %@", rfc822, 
newMail.mailbox.path );
> 		NSLog(@"lastError: %@", [msg.lastError localizedDescription]);
> 	}
> 
> Well, that suddenly hangs after about 1500 messages, and the rfc822 is 
only written for the couple first messages.
> 
> Today I tried to comment out the rfc822 part in my code, and boom, the 
scan is working fine. It looks like asking the rfc822 is quite costly on 
the server, as it just can't cope and just stop answering. So I'm thinking
I'll have to stick with the request of the body structure and get the 
rfc822 later in the code, or using a dedicated thread that checks the 
answer is here before I try the next messages.
> 
> Does that make sense?
> 
> Many thanks for the help,
> Olivier./.
> 
> 
> On Sep 20, 2012, at 5:32 PM, Olivier Palliere <olivier@molowa.com> wrote:
> 
>> Hi Matt and all,
>> 
>> I'll send the logs when XCode gives me the focus back (kinda colorwheel
of death right now ;-))
>> 
>> But form what I see, is it possible that whle replying to a FETCH with 
a very long video, other requests coming for a FETCH might be somehow 
queued and not giving back a value.
>> 
>> Form what I saw, I have a mail where a wmv is attached, and I keep getting 
>> 
>>>>>> send
>> 
>> 941 FETCH 937 RFC822
>> 
>>>>>>> end send
>> 
>> while I get some 
>> 
>> <<<<<< read
>> 
>> kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
>> elrgowe7gow837gopw397rghow3iurfgow]wigf
>> ;wurfghw3974ghpw374ghpw
>> 
>> ...
>> 
>> lots of data
>> 
>> ...
>> 
>> <<<<<< end read
>> 
>> then later again a 
>> 
>> 
>> <<<<<< read
>> 
>> kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
>> elrgowe7gow837gopw397rghow3iurfgow]wigf
>> ;wurfghw3974ghpw374ghpw
>> 
>> ...
>> 
>> lots of data
>> 
>> ...
>> 
>> <<<<<< end read
>> 
>> 
>> that could be the continuation of a previous FETCH, hence blocking me?
>> 
>> Many thanks in any case,
>> Olivier./.
>> 
>> On Sep 20, 2012, at 5:12 PM, Olivier Palliere <olivier@molowa.com> wrote:
>> 
>>> Hi Matt,
>>> 
>>> Will do. In the meantime, the lastError property on my CTCoreMessage 
is returning: Parse Error
>>> 
>>> 
>>> Olivier./.
>>> 
>>> On Sep 20, 2012, at 5:08 PM, Matt Ronge <mronge@mronge.com> wrote:
>>> 
>>>> Hi Olivier,
>>>> 
>>>> First try turning on MailCoreEnableLogging() and if you could please 
send a copy of the output from the console. That will give us an idea of 
if data is coming back properly from the server.
>>>> 
>>>> BTW: The log will contain passwords and other login information, so 
make sure to scrub that out
>>>> --
>>>> Matt Ronge
>>>> 
>>>> iOS Developer & Consultant
>>>> Central Atomics Inc.
>>>> http://www.centralatomics.com
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Sep 20, 2012, at 10:04 AM, Olivier Palliere <olivier@molowa.com> wrote:
>>>> 
>>>>> Hi Guys,
>>>>> 
>>>>> Still working on my mail client, I'm stumbling onto a weird problem.
I have to scan an account withe about 3800 messages.
>>>>> 
>>>>> Everything works fine in the beginning and for mailboxes that have a
limited number of messages. But when I'm on that big one, after a couple 
of messages, it start acting wrong.
>>>>> 
>>>>> The - rfc822 method always returns NULL, and thus, the string I'd 
like to save somewhere is also null.
>>>>> 
>>>>> Any idea where I should look?
>>>>> 
>>>>> I'm happy to give some code, but it won't be pretty ;-)
>>>>> 
>>>>> Many thanks,
>>>>> Olivier./.
>>>>> 
>>>> 
>>> 
>> 
> 

Re: [mailcore] CTCoreMessage rfc822

From:
Olivier Pallière
Date:
2012-09-21 @ 13:44
Hi!

I tried, but believe me it's obvious. I didn't see it in the first place 
because I was running it through a background thread so my app responsive.

But sampling the app shows that the scanning thread is hanging waiting for
a response from the socket connected to the server. 

I also noticed some I/O errors on the mail server side. 

Right now it's working really fast when I only request the envelope and 
the body, then later get the rfc on demand. 

And to be honest Instruments is kind of scary to me. I used DTrace on 
Solaris but I think I need a training for instruments ;-)

Many thanks,
Olivier./.

On Sep 21, 2012, at 3:35 PM, Jonathan Willing <jwilling@me.com> wrote:

> Hi,
> 
> You really need to run this through Instruments' profiler tool to see 
exactly what is holding it up. Making guesses about optimization is never 
a good idea.
> 
> Jonathan
> 
> On Sep 21, 2012, at 3:13 AM, Olivier Palliere <olivier@molowa.com> wrote:
> 
>> Hi Guys,
>> 
>> Further testing. This is for the initial scan of a mailbox, that has 
3800 messages. Right now, I'm getting the array of all messages using:
>> 
>>    NSArray *messageSet = [aFolder messagesFromSequenceNumber:1 to:0 
withFetchAttributes:CTFetchAttrEnvelope ];
>> 
>> 
>> Then looping through the array to get the infos I need for the messages:
>> 
>>      for (CTCoreMessage *msg in messageSet)
>>       {
>>           if ([self createMessage:msg 
inManagedObjectContext:localContext inCDMailbox:newBox])
>>           {
>>               NSLog(@"Message created (%ld)", currentStep);
>> 
>>           } else
>>           {
>>               NSLog(@"Error creating message %ld", currentStep);
>> 
>>           }
>> 
>> In the createMessage method, I assign all the fields to my CoreData 
entity using the CTCoreMessage:
>> 
>>      Messages *newMail = [Messages MR_createInContext:localContext]; //
Using MagicalRecord now
>> 
>>      [msg fetchBodyStructure];
>>      
>>       newMail.uid = [NSNumber numberWithInteger:msg.uid];
>>       newMail.messageID = msg.messageId;
>>          newMail.senderName = msg.sender.email;
>>    ....
>>    and so on
>> 
>> And at some point, I also ask the server to give me the rfc822 content 
for the message, to save it somewhere on disk
>> 
>>    NSString *rfc822 = msg.rfc822;
>>    NSError *writeError;
>>    if (![rfc822 writeToFile:localPath atomically:YES 
encoding:NSUTF8StringEncoding error:&writeError])
>>    {
>>        NSLog(@"error: %@", writeError.localizedFailureReason);
>>        NSLog(@"rfx822 : %@ NOTNOTNOTNOT saved for %@", rfc822, 
newMail.mailbox.path );
>>        NSLog(@"lastError: %@", [msg.lastError localizedDescription]);
>>    }
>> 
>> Well, that suddenly hangs after about 1500 messages, and the rfc822 is 
only written for the couple first messages.
>> 
>> Today I tried to comment out the rfc822 part in my code, and boom, the 
scan is working fine. It looks like asking the rfc822 is quite costly on 
the server, as it just can't cope and just stop answering. So I'm thinking
I'll have to stick with the request of the body structure and get the 
rfc822 later in the code, or using a dedicated thread that checks the 
answer is here before I try the next messages.
>> 
>> Does that make sense?
>> 
>> Many thanks for the help,
>> Olivier./.
>> 
>> 
>> On Sep 20, 2012, at 5:32 PM, Olivier Palliere <olivier@molowa.com> wrote:
>> 
>>> Hi Matt and all,
>>> 
>>> I'll send the logs when XCode gives me the focus back (kinda 
colorwheel of death right now ;-))
>>> 
>>> But form what I see, is it possible that whle replying to a FETCH with
a very long video, other requests coming for a FETCH might be somehow 
queued and not giving back a value.
>>> 
>>> Form what I saw, I have a mail where a wmv is attached, and I keep getting 
>>> 
>>>>>>> send
>>> 
>>> 941 FETCH 937 RFC822
>>> 
>>>>>>>> end send
>>> 
>>> while I get some 
>>> 
>>> <<<<<< read
>>> 
>>> kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
>>> elrgowe7gow837gopw397rghow3iurfgow]wigf
>>> ;wurfghw3974ghpw374ghpw
>>> 
>>> ...
>>> 
>>> lots of data
>>> 
>>> ...
>>> 
>>> <<<<<< end read
>>> 
>>> then later again a 
>>> 
>>> 
>>> <<<<<< read
>>> 
>>> kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
>>> elrgowe7gow837gopw397rghow3iurfgow]wigf
>>> ;wurfghw3974ghpw374ghpw
>>> 
>>> ...
>>> 
>>> lots of data
>>> 
>>> ...
>>> 
>>> <<<<<< end read
>>> 
>>> 
>>> that could be the continuation of a previous FETCH, hence blocking me?
>>> 
>>> Many thanks in any case,
>>> Olivier./.
>>> 
>>> On Sep 20, 2012, at 5:12 PM, Olivier Palliere <olivier@molowa.com> wrote:
>>> 
>>>> Hi Matt,
>>>> 
>>>> Will do. In the meantime, the lastError property on my CTCoreMessage 
is returning: Parse Error
>>>> 
>>>> 
>>>> Olivier./.
>>>> 
>>>> On Sep 20, 2012, at 5:08 PM, Matt Ronge <mronge@mronge.com> wrote:
>>>> 
>>>>> Hi Olivier,
>>>>> 
>>>>> First try turning on MailCoreEnableLogging() and if you could please
send a copy of the output from the console. That will give us an idea of 
if data is coming back properly from the server.
>>>>> 
>>>>> BTW: The log will contain passwords and other login information, so 
make sure to scrub that out
>>>>> --
>>>>> Matt Ronge
>>>>> 
>>>>> iOS Developer & Consultant
>>>>> Central Atomics Inc.
>>>>> http://www.centralatomics.com
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On Sep 20, 2012, at 10:04 AM, Olivier Palliere <olivier@molowa.com> wrote:
>>>>> 
>>>>>> Hi Guys,
>>>>>> 
>>>>>> Still working on my mail client, I'm stumbling onto a weird 
problem. I have to scan an account withe about 3800 messages.
>>>>>> 
>>>>>> Everything works fine in the beginning and for mailboxes that have 
a limited number of messages. But when I'm on that big one, after a couple
of messages, it start acting wrong.
>>>>>> 
>>>>>> The - rfc822 method always returns NULL, and thus, the string I'd 
like to save somewhere is also null.
>>>>>> 
>>>>>> Any idea where I should look?
>>>>>> 
>>>>>> I'm happy to give some code, but it won't be pretty ;-)
>>>>>> 
>>>>>> Many thanks,
>>>>>> Olivier./.
> 

Re: [mailcore] CTCoreMessage rfc822

From:
Jonathan Willing
Date:
2012-09-21 @ 13:48
I have a sneaking suspicion that you're trying to perform multiple I/O 
operations at the same time. Is there any chance you might be doing so, 
and encountering a deadlock?

On Sep 21, 2012, at 8:44 AM, Olivier Pallière <olivier@molowa.com> wrote:

> Hi!
> 
> I tried, but believe me it's obvious. I didn't see it in the first place
because I was running it through a background thread so my app responsive.
> 
> But sampling the app shows that the scanning thread is hanging waiting 
for a response from the socket connected to the server. 
> 
> I also noticed some I/O errors on the mail server side. 
> 
> Right now it's working really fast when I only request the envelope and 
the body, then later get the rfc on demand. 
> 
> And to be honest Instruments is kind of scary to me. I used DTrace on 
Solaris but I think I need a training for instruments ;-)
> 
> Many thanks,
> Olivier./.
> 
> On Sep 21, 2012, at 3:35 PM, Jonathan Willing <jwilling@me.com> wrote:
> 
>> Hi,
>> 
>> You really need to run this through Instruments' profiler tool to see 
exactly what is holding it up. Making guesses about optimization is never 
a good idea.
>> 
>> Jonathan
>> 
>> On Sep 21, 2012, at 3:13 AM, Olivier Palliere <olivier@molowa.com> wrote:
>> 
>>> Hi Guys,
>>> 
>>> Further testing. This is for the initial scan of a mailbox, that has 
3800 messages. Right now, I'm getting the array of all messages using:
>>> 
>>>   NSArray *messageSet = [aFolder messagesFromSequenceNumber:1 to:0 
withFetchAttributes:CTFetchAttrEnvelope ];
>>> 
>>> 
>>> Then looping through the array to get the infos I need for the messages:
>>> 
>>>     for (CTCoreMessage *msg in messageSet)
>>>      {
>>>          if ([self createMessage:msg 
inManagedObjectContext:localContext inCDMailbox:newBox])
>>>          {
>>>              NSLog(@"Message created (%ld)", currentStep);
>>> 
>>>          } else
>>>          {
>>>              NSLog(@"Error creating message %ld", currentStep);
>>> 
>>>          }
>>> 
>>> In the createMessage method, I assign all the fields to my CoreData 
entity using the CTCoreMessage:
>>> 
>>>     Messages *newMail = [Messages MR_createInContext:localContext]; //
Using MagicalRecord now
>>> 
>>>     [msg fetchBodyStructure];
>>> 
>>>      newMail.uid = [NSNumber numberWithInteger:msg.uid];
>>>      newMail.messageID = msg.messageId;
>>>         newMail.senderName = msg.sender.email;
>>>   ....
>>>   and so on
>>> 
>>> And at some point, I also ask the server to give me the rfc822 content
for the message, to save it somewhere on disk
>>> 
>>>   NSString *rfc822 = msg.rfc822;
>>>   NSError *writeError;
>>>   if (![rfc822 writeToFile:localPath atomically:YES 
encoding:NSUTF8StringEncoding error:&writeError])
>>>   {
>>>       NSLog(@"error: %@", writeError.localizedFailureReason);
>>>       NSLog(@"rfx822 : %@ NOTNOTNOTNOT saved for %@", rfc822, 
newMail.mailbox.path );
>>>       NSLog(@"lastError: %@", [msg.lastError localizedDescription]);
>>>   }
>>> 
>>> Well, that suddenly hangs after about 1500 messages, and the rfc822 is
only written for the couple first messages.
>>> 
>>> Today I tried to comment out the rfc822 part in my code, and boom, the
scan is working fine. It looks like asking the rfc822 is quite costly on 
the server, as it just can't cope and just stop answering. So I'm thinking
I'll have to stick with the request of the body structure and get the 
rfc822 later in the code, or using a dedicated thread that checks the 
answer is here before I try the next messages.
>>> 
>>> Does that make sense?
>>> 
>>> Many thanks for the help,
>>> Olivier./.
>>> 
>>> 
>>> On Sep 20, 2012, at 5:32 PM, Olivier Palliere <olivier@molowa.com> wrote:
>>> 
>>>> Hi Matt and all,
>>>> 
>>>> I'll send the logs when XCode gives me the focus back (kinda 
colorwheel of death right now ;-))
>>>> 
>>>> But form what I see, is it possible that whle replying to a FETCH 
with a very long video, other requests coming for a FETCH might be somehow
queued and not giving back a value.
>>>> 
>>>> Form what I saw, I have a mail where a wmv is attached, and I keep getting 
>>>> 
>>>>>>>> send
>>>> 
>>>> 941 FETCH 937 RFC822
>>>> 
>>>>>>>>> end send
>>>> 
>>>> while I get some 
>>>> 
>>>> <<<<<< read
>>>> 
>>>> kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
>>>> elrgowe7gow837gopw397rghow3iurfgow]wigf
>>>> ;wurfghw3974ghpw374ghpw
>>>> 
>>>> ...
>>>> 
>>>> lots of data
>>>> 
>>>> ...
>>>> 
>>>> <<<<<< end read
>>>> 
>>>> then later again a 
>>>> 
>>>> 
>>>> <<<<<< read
>>>> 
>>>> kdjfhgdgkfjgh9824ytpuwgfrigfieysgrfuiysegro
>>>> elrgowe7gow837gopw397rghow3iurfgow]wigf
>>>> ;wurfghw3974ghpw374ghpw
>>>> 
>>>> ...
>>>> 
>>>> lots of data
>>>> 
>>>> ...
>>>> 
>>>> <<<<<< end read
>>>> 
>>>> 
>>>> that could be the continuation of a previous FETCH, hence blocking me?
>>>> 
>>>> Many thanks in any case,
>>>> Olivier./.
>>>> 
>>>> On Sep 20, 2012, at 5:12 PM, Olivier Palliere <olivier@molowa.com> wrote:
>>>> 
>>>>> Hi Matt,
>>>>> 
>>>>> Will do. In the meantime, the lastError property on my CTCoreMessage
is returning: Parse Error
>>>>> 
>>>>> 
>>>>> Olivier./.
>>>>> 
>>>>> On Sep 20, 2012, at 5:08 PM, Matt Ronge <mronge@mronge.com> wrote:
>>>>> 
>>>>>> Hi Olivier,
>>>>>> 
>>>>>> First try turning on MailCoreEnableLogging() and if you could 
please send a copy of the output from the console. That will give us an 
idea of if data is coming back properly from the server.
>>>>>> 
>>>>>> BTW: The log will contain passwords and other login information, so
make sure to scrub that out
>>>>>> --
>>>>>> Matt Ronge
>>>>>> 
>>>>>> iOS Developer & Consultant
>>>>>> Central Atomics Inc.
>>>>>> http://www.centralatomics.com
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Sep 20, 2012, at 10:04 AM, Olivier Palliere <olivier@molowa.com> wrote:
>>>>>> 
>>>>>>> Hi Guys,
>>>>>>> 
>>>>>>> Still working on my mail client, I'm stumbling onto a weird 
problem. I have to scan an account withe about 3800 messages.
>>>>>>> 
>>>>>>> Everything works fine in the beginning and for mailboxes that have
a limited number of messages. But when I'm on that big one, after a couple
of messages, it start acting wrong.
>>>>>>> 
>>>>>>> The - rfc822 method always returns NULL, and thus, the string I'd 
like to save somewhere is also null.
>>>>>>> 
>>>>>>> Any idea where I should look?
>>>>>>> 
>>>>>>> I'm happy to give some code, but it won't be pretty ;-)
>>>>>>> 
>>>>>>> Many thanks,
>>>>>>> Olivier./.
>>