Servio stops updating libraries

Issue #1054 resolved
Eric Courtwright created an issue

Not sure what's happened. Running for over 2 years. after running between 24 and 48 hours, the library stops updating.

When I stop the serviio service, it hangs. I typically have to kill the service using taskkill or reboot the system

Let me know what logs and details you need.

Running on a windows 2016 server with 4TB library (6 900gb drives, in Raid 5) 12 2.5ghz xeon intel cores.

Took at look through the logs, but I'm not really seeing anything. I did try deleting the library directory, and allowing it to rebuild, which didn't help.

Let me know where to go from here. Thanks

Comments (44)

  1. Petr Nejedly repo owner
    • removed version

    check the log to see what is the last added file, it might be corrupted and blocking further progress. Try to remove the file.

  2. Eric Courtwright reporter

    it's stopped multiple times,-- been having to restart it every 2 days for the last month.
    It's not a single file causing the issue. I have a debug log now, but I've not really been able to find anything that looks like a problem in it..

    this seemed to start when the server went from 2012 to 2016

  3. Eric Courtwright reporter

    when I notice it's not listing new stuff. I stop the serviio service, then kill it, (because stop hangs) give it a couple minutes, the restart the service. New stuff typically shows up in seconds. So don't seem to be the same issue

  4. Eric Courtwright reporter

    I have a log, problem is, with debug on, logs are large, and it may be several hours before I notice it's not updating. I can't find anything in the log I have.

  5. Petr Nejedly repo owner

    It might be a side effect of something I fixed for 1.9.2.

    What are these devices?

    #!
    
    2018-03-20 23:34:16,690 DEBUG [RendererManager] Retrieve device description from http://10.10.222.65:7676/smp_11_
    2018-03-20 23:34:16,831 DEBUG [RendererManager] Retrieve device description from http://10.10.222.70:7676/smp_18_
    2018-03-20 23:34:16,844 DEBUG [RendererManager] Retrieve device description from http://10.10.222.70:7676/smp_18_
    2018-03-20 23:34:16,851 DEBUG [RendererManager] Retrieve device description from http://10.10.222.70:7676/smp_18_
    2018-03-20 23:34:16,858 DEBUG [RendererManager] Retrieve device description from http://10.10.222.65:7676/smp_11_
    

    Can you connect to these URLs? I suspect they block or take ages and that might mean Serviio runs out of available threads over time.

  6. Eric Courtwright reporter

    listed it in the first post.. windows 2016 server. and to be honest, the 2012 to 2016 server upgrade is about when this started (but not exactly). I'll give it a shot.

  7. Petr Nejedly repo owner

    post the log pls. btw, how do you know, do you just add a new file and it doesn't show up? are you using Poller or native scanning for the shared folders? Is the service stuck now?

  8. Eric Courtwright reporter

    SHOOT, it was late, and I forgot to grab the log before restarting. It did hang on stopping the service. In task manager, there were 2 serviio tasks, one is using less than 1mb of ram, the other about 169 megs.

    Yes, that's exactly what happens. Files are added, and don't show up. A few weeks ago, I had tried turning polling on, to see if it helped. It did not, so it remains off. These disks are local disks, (and dang fast), so from the description, it didn't seem needed.

    After Serviio restart, the files typically show up within a few seconds. I will grab the log on the next hang. I'll also try to keep closer watch on it, so maybe I can narrow down the problem area in the log a bit.

  9. Eric Courtwright reporter

    I have an alarm set every hour... I move a new file out, confirm it shows, and then delete, and confirm it's removed.

    it updates in less than 5 seconds, every time right now.

  10. Eric Courtwright reporter

    OK-- after many hours of testing.
    It died between 1:45 and 2:25 am... supernatural was added at 1:46 am, and Arrow at 10:06 PM

    so the process failed sometime between then

  11. Petr Nejedly repo owner

    The attached log file is not using debug logging. Not much to see there unfortunately.

  12. Eric Courtwright reporter

    dont forget to put it in debug. I've been out of the country the last 2 weeks, just got back yesterday, so I'll try to catch a log too.

  13. Eric Courtwright reporter

    Peter > yes, I had tried force refresh after the refresh stops, and it doesn't work.

  14. gsrbiker

    Hello, I think I got something; over 24 hours after restarting serviio with debug logging, I dropped a new file into a monitored folder: this two messages gets printed over and over and over, 4 or 5 times per second.

    2018-04-18 10:10:55,681 DEBUG [TheTVDBv2SourceAdaptor] Successfully logged into thetvdb.com 2018-04-18 10:10:55,683 DEBUG [TheTVDBv2SourceAdaptor] Token for thetvdb.com is invalid or expired

    This went on for four minutes before stopping.

    Do you need the whole log file?

    Best regards, Silla

  15. Eric Courtwright reporter

    I just had the updates stop. I went in and hit force-refresh, and it WORKED. Is it possible this is new since 1.9.2 ? because before when it hung, force-refresh wouldn't work. I hadn't tried that since the update.

  16. tubularusa

    Same issue on Synology (DSM 6.1.6) running Serviio 1.9.1. Newly added video files stop being added to the library. The only way to force a refresh is to uncheck "keep library automatically updated" modify the shared video folder to something different, hit Save, it will hang, refresh the browser, then the folder will reverts back by itself, recheck "keep library automatically updated," and hit Save. It will start scanning everything. But at some random point within the next few day(s), it will stop refreshing again. Here is my latest log showing where I added a new video file "2018-04-18 10:10:26,478 DEBUG [PipeManager] Adding file '/volume1/video/Series/The Crossing/The Crossing.S01E03.480p[FromAnime].mkv' to the delta queue (file added)."

    serviio.log

    Even thought it knows I'm trying to add the file, the library never refreshed to show this new video afterwards.

  17. gsrbiker

    Sorry, my previous report was most likely not significant. The file has been added to the library.

  18. Petr Nejedly repo owner

    @tubularusa could you post detailed log of (re)starting the server and the following 1 minute of running? also do you have the older logs still ? serviio.log.0, etc

  19. Petr Nejedly repo owner

    @gsrbiker thanks, I think that's the problem - it was stuck on trying to fetch the online metadata and just in a loop. please test once 1.9.2 is in beta (send me a PM on the forum if you are not a tester yet)

  20. Eric Courtwright reporter

    His issue may be solved, but mine isn't. It stopped updating sometime last night, but with the logs in debug, I can't tell if there's anything in there or not, as there's quite a few hours between when it stopped, and when I noticed it.

    I'll look through the logs to see if I can tell...

  21. Eric Courtwright reporter

    yeah, it's currently 5:29 (17:29), oldest log only goes back to 12:03.

    I got plenty of disk space, any way to extend logging ?

    edit: found the <param name="MaxBackupIndex" value="5" /> reset for <param name="MaxBackupIndex" value="25" />

    should hold at least 24 hours that way.

    Restarting now, I'm sure it will hang, as it always does.

  22. Petr Nejedly repo owner

    I mean I've fixed it (I think) for 1.9.2, which is not released yet. I'm pretty confident it's the same problem for both of you.

  23. Eric Courtwright reporter

    I've been running 1.92 since 2018-03-23

    unless it's an update since then, I still see it.

  24. Petr Nejedly repo owner

    @ecourtt you have an early 1.9.2 build. I fixed this only yesterday. It'll be a part of an official beta on the forums.

  25. Log in to comment