Crash when referencing non-existent file

Issue #759 resolved
Gili created an issue

I recreated a repository behind Rhodecode's back. When I tried accessing it, I got the following exception which caused the entire server to shut down:

Error - <type 'exceptions.IOError'>: [Errno 2] C:\Users\Gili\Documents\MercurialRepositories\webrtc\.hg/store\00changelog.d: The system cannot find the file specified
URL: https://bbs.darktech.org/hg/webrtc/summary
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\weberror\\errormiddleware.py', line 162 in __call__
  app_iter = self.application(environ, sr_checker)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\beaker\\middleware.py', line 155 in __call__
  return self.wrap_app(environ, session_start_response)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\routes\\middleware.py', line 131 in __call__
  response = self.app(environ, start_response)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\pylons\\wsgiapp.py', line 107 in __call__
  response = self.dispatch(controller, environ, start_response)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\pylons\\wsgiapp.py', line 312 in dispatch
  return controller(environ, start_response)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\rhodecode\\lib\\base.py', line 300 in __call__
  return WSGIController.__call__(self, environ, start_response)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\pylons\\controllers\\core.py', line 211 in __call__
  response = self._dispatch_call()
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\pylons\\controllers\\core.py', line 162 in _dispatch_call
  response = self._inspect_call(func)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\pylons\\controllers\\core.py', line 105 in _inspect_call
  result = self._perform_call(func, args)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\pylons\\controllers\\core.py', line 57 in _perform_call
  return func(**args)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\rhodecode\\controllers\\summary.py', line 141 in index
  run_task(get_commits_stats, c.dbrepo.repo_name, ts_min_y, ts_max_y)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\rhodecode\\lib\\celerylib\\__init__.py', line 79 in run_task
  return ResultWrapper(task(*args, **kwargs))
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\celery\\task\\base.py', line 241 in __call__
  return self.run(*args, **kwargs)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\celery\\app\\__init__.py', line 141 in run
  return fun(*args, **kwargs)
File '<string>', line 2 in get_commits_stats
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\rhodecode\\lib\\celerylib\\__init__.py', line 122 in __wrapper
  ret = func(*fargs, **fkwargs)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\rhodecode\\lib\\celerylib\\tasks.py', line 154 in get_commits_stats
  k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\rhodecode\\lib\\vcs\\utils\\lazy.py', line 27 in __get__
  result = obj.__dict__[self.__name__] = self._func(obj)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\rhodecode\\lib\\vcs\\backends\\hg\\changeset.py', line 56 in date
  return date_fromtimestamp(*self._ctx.date())
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\context.py', line 185 in date
  return self._changeset[2]
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\util.py', line 246 in __get__
  result = self.func(obj)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\context.py', line 138 in _changeset
  return self._repo.changelog.read(self.rev())
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\changelog.py', line 279 in read
  text = self.revision(node)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\revlog.py', line 909 in revision
  self._chunkraw(base, rev)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\revlog.py', line 828 in _chunkraw
  return self._getchunk(start, length)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\revlog.py', line 821 in _getchunk
  return self._loadchunk(offset, length)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\revlog.py', line 798 in _loadchunk
  df = self.opener(self.datafile)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\store.py', line 448 in __call__
  return self.vfs(self.encode(path), mode, *args, **kw)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\scmutil.py', line 317 in __call__
  fp = util.posixfile(f, mode)
File 'c:\\Users\\Gili\\Documents\\rhodecode\\lib\\site-packages\\mercurial\\windows.py', line 35 in posixfile
  raise IOError(err.errno, '%s: %s' % (name, err.strerror))
IOError: [Errno 2] C:\Users\Gili\Documents\MercurialRepositories\webrtc\.hg/store\00changelog.d: The system cannot find the file specified


CGI Variables
-------------
  CONNECTION_TYPE: 'Keep-Alive'
  CONTENT_TYPE: '; charset="utf-8"'
  HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
  HTTP_ACCEPT_CHARSET: 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
  HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch'
  HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.8'
  HTTP_COOKIE: 'atlassian.xsrf.token=A6N4-FM5X-KNGU-B4D4|d4b64001c2a8388ce399fefa7ff3bdfc6d8403a8|lout; AJS.conglomerate.cookie=""; JSESSIONID=EE688F5EBACA3694B56E8E9DD17281C5; rhodecode=5eb47898b5004f8292f62893dd52463a; doc-sidebar=300px'
  HTTP_HOST: 'bbs.darktech.org'
  HTTP_REFERER: 'https://bbs.darktech.org/hg/'
  HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17'
  PATH_INFO: '/webrtc/summary'
  REMOTE_ADDR: '192.168.1.1'
  REQUEST_METHOD: 'GET'
  SCRIPT_NAME: '/hg'
  SERVER_NAME: 'bbs.darktech.org'
  SERVER_PORT: '1003'
  SERVER_PROTOCOL: 'HTTP/1.1'
  SERVER_SOFTWARE: 'waitress'


WSGI Variables
--------------
  application: <beaker.middleware.SessionMiddleware object at 0x000000000453FA20>
  beaker.get_session: <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x000000000453FA20>>
  beaker.session: {'_authentication_token': '9c9d5222b56cf33c4ab0bd4a8e5304399cecf084', 'rhodecode_user': {'username': u'gili.tzabari', 'is_authenticated': True, 'user_id': 2L}, '_accessed_time': 1360252359.119, '_creation_time': 1360014272.657}
  paste.registry: <paste.registry.Registry object at 0x0000000005F3EA20>
  paste.throw_errors: True
  pylons.action_method: <bound method SummaryController.index of <rhodecode.controllers.summary.SummaryController object at 0x0000000005F444A8>>
  pylons.controller: <rhodecode.controllers.summary.SummaryController object at 0x0000000005F444A8>
  pylons.environ_config: {'session': 'beaker.session', 'cache': 'beaker.cache'}
  pylons.pylons: <pylons.util.PylonsContext object at 0x00000000062AFCC0>
  pylons.routes_dict: {'action': u'index', 'controller': u'summary', 'repo_name': u'webrtc'}
  routes.cached_hostinfo: {'host': 'bbs.darktech.org', 'protocol': 'https'}
  routes.route: <routes.route.Route object at 0x0000000003FCCD68>
  routes.url: <routes.util.URLGenerator object at 0x0000000005FE40F0>
  webob._parsed_query_vars: (GET([]), '')
  webob.adhoc_attrs: {'user': <AuthUser('id:2:gili.tzabari|True')>, 'language': 'en-us'}
  wsgi process: 'Multithreaded'
  wsgi._org_proto: 'http'
  wsgi.file_wrapper: <class 'waitress.buffers.ReadOnlyFileBasedBuffer'>
  wsgiorg.routing_args: (<routes.util.URLGenerator object at 0x0000000005FE40F0>, {'action': u'index', 'controller': u'summary', 'repo_name': u'webrtc'})
------------------------------------------------------------
2013-02-07 10:52:39.288 INFO  [rhodecode.lib.base] IP: 192.168.1.1 User: <AuthUser('id:2:gili.tzabari|True')> accessed /webrtc/summary
2013-02-07 10:52:39.290 INFO  [rhodecode.lib.base] IP: 192.168.1.1 Request to /webrtc/summary time: 0.019s
2013-02-07 10:52:40.266 INFO  [rhodecode.lib.base] IP: 192.168.1.1 User: <AuthUser('id:2:gili.tzabari|True')> accessed /webrtc/summary
2013-02-07 10:52:40.267 INFO  [rhodecode.lib.auth] user gili.tzabari is authenticated and granted access to SummaryController:__before__ using RegularAuth
2013-02-07 10:52:40.334 INFO  [rhodecode.lib.celerylib.tasks] running task with lockkey task_caca57a85507a92b4631b11de56d4853.lock

Comments (8)

  1. Marcin Kuzminski repo owner

    well it's a case of really damaged repository if it's missing mercurial files in the store, but i tried to reproduce this issue on my local env, and it just shows such repos as empty, is it always reproducible on your side (eg removing this file from repos always shutdowns the server ?)

  2. Gili reporter

    That's not what I meant.

    I had a repository called "webrtc" with some history. I deleted the directory without informing Rhodecode and re-created another repository (with a slightly different history) in the same directory. Some changesets were removed. Some changesets were added. Neither repository was damaged.

    When I attempted to access the repository using Rhodecode, it caused the above crash. Unfortunately I did not keep the original repository so I can no longer reproduce the issue.

  3. Marcin Kuzminski repo owner

    Ok so doing some mangling on repositories without telling it to RhodeCode can lead to some issues, but there's simple solution to that, next time you do this go into admin pannel of RhodeCode and run rescan repos command it should just catch up with new data, and invalidate any caches of repos that has been changed "outside" of RhodeCode

  4. Gili reporter

    I understand, but rhodecode should not crash on such changes. Ideally detecting such changes should automatically trigger a rescan on the repository in question.

  5. Marcin Kuzminski repo owner

    I agree it shouldn't ! :)

    Are you sure it was not memory error or something that killed the RhodeCode process during this error ? I'll try to reproduce this somehow, but without reproducible example it's going to be hard to fix or check exactly why it died.

  6. Gili reporter

    Looking at the log, try killing .hg/store\00changelog.d and accessing the summary page of the repository. It should trigger a crash.

  7. Log in to comment