Ruby - Threads and Dir[] arrays -

-

i have built ruby script creates folders, moves file new folder , invokes system() call trigger ffmpeg. have turned 4 threads can 4 concurrent transcodes @ time.

here example 2 threads (minus folder structure creation , file move functions):

def transcode_process_1   dir["path/file/source/*.mxf"].each |f|     random_folder = #code random folder creation     file_move = #code move .mxf file random_folder processing     system("ffmpeg -i #{random_folder} command 2> /path/file/#{random_filename}.txt")     sleep(2)   end end  def transcode_process_2   sleep(3)   dir["path/file/source/*.mxf"].each |f|     random_folder = #code random folder creation     file_move = #code move .mxf file random_folder processing     system("ffmpeg -i #{random_folder} command 2> /path/file/#{random_filename}.txt")     sleep(4)   end end  transcode_thread_1 = thread.new{transcode_process_1()} transcode_thread_2 = thread.new{transcode_process_2()}  transcode_thread_1.join transcode_thread_2.join

this iterates through dir "path/file/source/" , processes .mxf files finds. issue having when both threads running adding same files array. results in ffmpeg stating cannot locate file (this because thread has processed , moved temp folder) , creating superfluous folders , log files, making messy.

my question how go making sure transcode_thread_2 not process files transcode_thread_1 has added array? there way can function check file in array still exists? if carry out process, if not move on next file?

thanks

there 2 ways fix this, consider second way superior.

1 - check if file exists before proceeding

simply add file.exist?(f) check code:

def transcode_process_1     dir["path/file/source/*.mxf"].each |f|         next if !file.exist?(f)         ..carry on normal..

2 - use queue (part of threads package)

you can use queue, accessible threads. simplifies things in number of ways

  1. central list of items process
  2. accessible threads, can spawn many threads can handle
  3. removes multiple versions of same code , code smell sleep values

something along lines of:

require 'thread'  total_threads = 2 # or cores have      transcode_queue = queue.new transcode_queue << dir["path/file/source/*.mxf"]  def transcode_process(queue)     while !queue.empty?         file = queue.pop # process file          random_folder = '' #code random folder creation         random_filename = '' # missing example          file_move = '' #code move .mxf file random_folder processing          system("ffmpeg -i #{random_folder} command 2> /path/file/#{random_filename}.txt")     end end  (1..total_threads).each {     thread.new { transcode_process(transcode_queue) } }

you don't need sleep() there should no race condition anymore having accessed queue, , you've shared same code each new thread.


Comments

Post a Comment

Popular posts from this blog

java - Run spring boot application error: Cannot instantiate interface org.springframework.context.ApplicationListener -

-
i have spring project , try make use spring boot , run @ embadded tomcat following : https://spring.io/guides/gs/rest-service/ this application //@configuration //@enableaspectjautoproxy @springbootapplication @componentscan(basepackages = "gux.prome") public class application { public static void main(string[] args) { springapplication.run(application.class, args); } } if use maven command: mvn spring-boot:run , project starts fine, need debug , run main method @ intelij, exception occurs: exception in thread "main" java.lang.illegalargumentexception: cannot instantiate interface org.springframework.context.applicationlistener : org.springframework.boot.logging.classpathloggingapplicationlistener @ org.springframework.boot.springapplication.createspringfactoriesinstances(springapplication.java:414) @ org.springframework.boot.springapplication.getspringfactoriesinstances(springapplication.java:394) @ org.springframework.bo...
Read more

reactjs - React router and this.props.children - how to pass state to this.props.children -

-
i'm using react-router first time , don't know how think in yet. here's how i'm loading components in nested routes. entry point .js reactdom.render( <router history={hashhistory} > <route path="/" component={app}> <route path="models" component={content}> </route> </router>, document.getelementbyid('app') ); app.js render: function() { return ( <div> <header /> {this.props.children} </div> ); } so child of app content component sent in. i'm using flux , app.js has state , listens changes, don't know how pass state down this.props.children. before using react-router app.js defines children explicitly, passing state natural don't see how now. using couple of react helper methods can add state, props , whatever else this.props.children render: function() { var children = react.ch...
Read more

Excel VBA "Microsoft Windows Common Controls 6.0 (SP6)" Location Changes -

-
i have excel workbook used generate quotations customers. there main page user inputs data, , there buttons generate various pdfs , send them customer. however, 1 user consistently getting error preventing generating , sending pdfs. i beleive "microsoft windows common controls 6.0 (sp6)". it's installed on users' computers. on mine, location "c:\windows\syswow64\mscomctl.ocx" on others "c:\windows\system32\mscomctl.ocx" the user experiencing issues on 32-bit machine, "mscomctl.ocx" file should in system32. have manually put in there, , pointed common controls reference in vba it. issue seems keep reverting looking in syswow64. since folder doesn't exist, can't find it, , can't produce pdfs. i thought perhaps issue user running older version of office (2007) there users on 2013 , 2016. thought perhaps mixture of 64-bit , 32-bit users. user gets sent workbook other users (as quotation process goes further, gets passed ...
Read more