log in

Advanced search

Message boards : News : Faster Download Speeds

Author Message
Profile Tom
Avatar
Send message
Joined: 23 Jun 08
Posts: 490
Credit: 238,767
RAC: 0
Message 1862 - Posted: 28 Jun 2011, 15:23:05 UTC
Last modified: 28 Jun 2011, 15:23:43 UTC

In our previous batch of workunits, volunteers reported download speeds ranging from 14kB/s to 100kB/s when pulling down work from our servers. We've since updated our distribution system so individual workunits will be orders of magnitudes smaller, and thus download speeds should be greatly increased. If you volunteer your computer for the current batch of workunits and are still experiencing problems, please let us know and we will diagnose the issue further.

Also, the initial workunit may take longer than usual to download since we have a new application version to distribute, but all subsequent downloads should be much quicker.

Profile trigggl
Send message
Joined: 10 Mar 09
Posts: 28
Credit: 103,390
RAC: 0
Message 1863 - Posted: 28 Jun 2011, 17:35:10 UTC - in response to Message 1862.

My downloads are 4k just like in May.

I do appreciate the effort, though.

Profile Tom
Avatar
Send message
Joined: 23 Jun 08
Posts: 490
Credit: 238,767
RAC: 0
Message 1864 - Posted: 28 Jun 2011, 18:56:53 UTC - in response to Message 1863.
Last modified: 28 Jun 2011, 21:09:30 UTC

That is unfortunate. However, since most workunits will be less than 100kB in size (as opposed to 10MB) you should notice that you are able to download more workunits per hour and contribute more to the project.

As mentioned in the initial post, the application files are still about the same size (500kB-35MB), but those will only be downloaded as often as we release a new application.

(EDIT: application file sizes)

reklov
Send message
Joined: 3 Feb 11
Posts: 4
Credit: 102,518
RAC: 0
Message 1865 - Posted: 28 Jun 2011, 20:03:31 UTC - in response to Message 1864.

I'm now downloading for more han 3 hours for the 1st wu and still have more than 17MB to go. (actual file sbd_1.41_windows_intelx86.core has 29,86MB, the size of the other files you'll know are much more than the mentioned 1MB)

reklov
Send message
Joined: 3 Feb 11
Posts: 4
Credit: 102,518
RAC: 0
Message 1866 - Posted: 28 Jun 2011, 20:03:46 UTC - in response to Message 1864.
Last modified: 28 Jun 2011, 20:05:00 UTC

[duplicate post deleted]

zombie67 [MM]
Avatar
Send message
Joined: 25 Jan 08
Posts: 89
Credit: 4,496,276
RAC: 0
Message 1867 - Posted: 28 Jun 2011, 20:21:53 UTC

Can the number of tasks in process for a host be increased from the current setting of 1?
____________
Reno, NV
Team: SETI.USA

Profile Tom
Avatar
Send message
Joined: 23 Jun 08
Posts: 490
Credit: 238,767
RAC: 0
Message 1869 - Posted: 28 Jun 2011, 21:27:33 UTC - in response to Message 1867.
Last modified: 28 Jun 2011, 21:28:36 UTC

I apologize that the application files are still taking a long time to download. We've addressed the issue for individual workunits, but we'll need to look at more systemic causes for why our servers cannot handle large downloads volumes. This should only be a concern when we push out new application versions, but it's enough of a bottleneck that we will continue to devote our time and energy to resolving the issue.

Also, zombie67, I've removed the limit on the max number of workunits in progress.

zombie67 [MM]
Avatar
Send message
Joined: 25 Jan 08
Posts: 89
Credit: 4,496,276
RAC: 0
Message 1870 - Posted: 28 Jun 2011, 21:36:59 UTC - in response to Message 1869.
Last modified: 28 Jun 2011, 22:10:10 UTC

Thanks!

Edit: Out of work already? More to come?
____________
Reno, NV
Team: SETI.USA

Profile Tom
Avatar
Send message
Joined: 23 Jun 08
Posts: 490
Credit: 238,767
RAC: 0
Message 1873 - Posted: 29 Jun 2011, 14:24:49 UTC - in response to Message 1870.

It will probably be at least several weeks before a significant number of workunits are generated, but there may be some occasional small jobs in between.

frankhagen
Send message
Joined: 22 Oct 08
Posts: 21
Credit: 71,448
RAC: 0
Message 1874 - Posted: 29 Jun 2011, 15:45:13 UTC - in response to Message 1873.
Last modified: 29 Jun 2011, 15:59:46 UTC

i suggest you pack it all together using 7z http://www.7-zip.org/ :



7-Zip 4.65 Copyright (c) 1999-2009 Igor Pavlov 2009-02-03

Listing archive: sbcl_1.41

Method = LZMA
Solid = -
Blocks = 1
Physical Size = 5958480
Headers Size = 164

Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2011-06-29 17:41:15 ....A 31313944 5958316 sbcl_1.41_windows_intelx86
core

ports are available for MAC-OSX, several linux branches, AIX, Solaris...

just in case anyone not happy - source-code is available too!


could save a LOT of bandwith for all of us..

Profile trigggl
Send message
Joined: 10 Mar 09
Posts: 28
Credit: 103,390
RAC: 0
Message 1875 - Posted: 29 Jun 2011, 18:10:14 UTC - in response to Message 1873.

It will probably be at least several weeks before a significant number of workunits are generated, but there may be some occasional small jobs in between.

So, what we just had, is that considered a significant number or a small job?

reklov
Send message
Joined: 3 Feb 11
Posts: 4
Credit: 102,518
RAC: 0
Message 1876 - Posted: 29 Jun 2011, 18:32:54 UTC - in response to Message 1869.

Yesterday I got one single wu, runtime less than 20 minutes, but download time overall was much more than 4 hours. Maybe you should give credits per download time instead of calculation time, I would have gotten more than the 7cr for this wu. I hope, next time no new application is needed.

Profile Tom
Avatar
Send message
Joined: 23 Jun 08
Posts: 490
Credit: 238,767
RAC: 0
Message 1877 - Posted: 1 Jul 2011, 19:40:17 UTC

Decreasing the size of the files we distribute is a great way to tackle the issue (especially given the compression ratio for the sbcl core!), but we also need to troubleshoot the core factor that's limiting our transfer speeds. This will allow us to better scale the system in the future.

As for batch sizes, I consider anything less than 100,000 workunits to be fairly small for any type of grid resource, but that ultimately depends on how long each workunit runs. To answer your question directly 7ri9991, most of our recent batches have been insignificant in size.

And finally, I (strongly) don't perceive that the next round of workunits will require an update to our application, but I can't say definitively.

Message boards : News : Faster Download Speeds


Main page · Your account · Message boards


Copyright © 2023 MindModeling.org