 |
Maybe you can explain to me how multi-threading will make things go faster. The accurate code fetches the values from the database and sends them over the socket to the filed. Then its stored into a memory hash table or into an LMDB for bareos-14.2. So what will multi-threading bring unless you open multiple sockets etc. e.g. you have a resource starvation on the socket. The only thing you could think about is instead of sending the items one by one send them as bigger chunks or use some socket compression. |
|
|
Sorry if not enough clear and especially using certainly wrong words, for non dev language translated to dev. Don't shoot the fool :-) So from your point there's no way to check if files have changed in multi-* process? if file changed -> put them on the tobedone-queue. I guess that checksuming 4 to 8 file at the same time result in a whole shorter backup time, than doing the checksum one by one. Or to check where the time is most spended, what would be the procedure to see how much time is needed to build the accurate information on dir, the time spended to send it to -fd, the time to build the hash/lmdb the time spended to locally build what as to be backuped. |
|
|
If you want to see where time is spend you need to use profiling. It might be worthwhile to split the scan process but that means decoupling the scan process done in the findlib and the actual saving of the data. It may also mean we need more read only transactions for the LMDB as multiple threads will be accessing the accurate data. Same is true for the in memory hash as multiple threads may be updating the data. Given our current workload don't think we will be spending much time on this but if you want to try fork the code and try it out not sure how hard it will be. |
|
|
New work is in progress for 23. |
- Anonymous
