photo

Smith

shared this idea
3 years ago

Employees Involved

photo

SCM

Admin

Statistics

12
Comments
1
Views

Share

82
votes

Multiple threads

Hi, do you think it would be possible to add multiple thread support? That way when you add say 10 proxies you could do 5 or 10 threads and the program would run way faster... Thanks!

Official Answer
photo Employee
SCM Posted 3 years ago

4.16.5.24 - Under Options -> Advanced settings, you can change number of threads.

This is *unsupported* officially.

Add Comment

Comments (12)

photo Employee
50

SCM does multithread all scraper tasks. The only thing it doesn't is when scraping Google/Bing. It slows down for that.

photo
61

The Article Creator Tasks could be multithreaded. Right now in Article Generator tab if i add multiple article creator jobs it takes one at a time and all the rest are waiting for the previous one to finish before running. That would be way faster if we could run multiple tasks in same time and be able to tie each task to diffrent IPs.

photo Employee
53

Claudia wrote:

The Article Creator Tasks could be multithreaded. Right now in Article Generator tab if i add multiple article creator jobs it takes one at a time and all the rest are waiting for the previous one to finish before running. That would be way faster if we could run multiple tasks in same time and be able to tie each task to diffrent IPs.
Yes you are correct here.

I will consider enabling for users.

photo
50

Hi,

Thanks for adding that, but how is it exactly working? Because i went to Options set it to 5 threads, created 5 tasks each with one keyword, then if i start all 5 tasks is still just running one task at a time...

photo Employee
45

Claudia wrote:

Hi,

Thanks for adding that, but how is it exactly working? Because i went to Options set it to 5 threads, created 5 tasks each with one keyword, then if i start all 5 tasks is still just running one task at a time...

You should restart SCM at least once (forgot to mention)

photo
44

Yes thanks is working now.

photo
49

Tested this and it crashes basically all the time when running multiple threads. It just auto closes itself at random times. The only way to not crash is to run one thread at a time.

photo Employee
54

Claudia wrote:

Tested this and it crashes basically all the time when running multiple threads. It just auto closes itself at random times. The only way to not crash is to run one thread at a time.
Are running this on desktop or via rdp on a VPS?

photo
45

I'm running it on my desktop computer using a VPN service, which works great when running one thread. Tested today and it works great when running 2 threads as well, but if you run like 4 threads for example, then it crashes at random times.

photo Employee
47

Claudia wrote:

I'm running it on my desktop computer using a VPN service, which works great when running one thread. Tested today and it works great when running 2 threads as well, but if you run like 4 threads for example, then it crashes at random times.
SCM was never tested with multi-thread support. If it crashes slightly its harder to figure out what is wrong.

My guess is memory overflow errors or something in the threaded code not working as expected.

photo
52

Yeah that's not very encouraging. Releasing a feature that was not tested and not working as supposed too. Why release it at all then? LOL

Tested here too and can confirm this feature is not working. Crashes all the time

photo
13

This needs serious reconsideration especially because producing articles from bing searches is VERY slow. Either that, or allow more threads for searching bing/google when one task is running. For example I have 100 proxies, but cannot use them because I'm not able to increase threads in SEO CM. This is a must.

Leave Comment

photo