API sends "304 Not Modified" for pages that have been modified

[Kyle Lanchman]

You can use the following curl command for testing:

curl -H "If-None-Match: \"anime/cast/32948\"" -i http://localhost:8000/2.1/anime/cast/32948

Steps to reproduce

• Open an editor. You'll need to have Controllers/RecordController.php and Models/Cast.php open.
• For ease, change the max age for the response to 30 seconds instead of 86400 seconds
• Run the server in prod mode: php app/console server:run --env=prod
• Delete the prod cache folder
• Do the following within 30 seconds total
  • curl
    • Observe output. Should get 200 OK.
  • curl again
    • Observe output. Should get 304 Not Modified
  • Change setName(...) in Cast.php to set a hardcoded string as the cast member's name
  • curl again
    • Observe output. Should get 304 Not Modified
• Wait for 30 seconds
• curl again
  • Observe that the request took longer to process (it built a new response as expected). Observe output. We should get a new response with the new hardcoded names, but instead we get a 304 Not Modified.

Problem: The server is directing the client to use local cached data even in cases where there have been modifications.

Proposed solution

I looked at the Symfony documentation for expiration caching and made the following changes:

• Remove the $response->setEtag(...) call
• Change $response->setMaxAge(...) to $response->setSharedMaxAge(...)

When running curl against this configuration, remove the If-None-Match header. You will receive 200 OK for all requests. You'll notice that the server sends cached responses for 30 seconds before building a new response. You can verify by following the same process of hardcoding a name in Cast.php.

Caveat: The server will send a full response every time rather than an empty 304 Not Modified, which will use more bandwidth. I'd say this is better than sending incorrect information, though.

Notes

This is likely an issue anywhere $response->setEtag(...) is used!

I have been testing against the latest commit on master, but I was also able to reproduce the issue on develop when I enabled caching by uncommenting line 13 in web/app.php.

---

Hopefully what I wrote is easy to follow. Please let me know if you have any questions!

[Michael Johnson]

The idea for caching direction is to keep bandwidth, requests to MAL, and other resource usage down for endpoints where getting fresh data is not critical. Sending a full response would be somewhat counter to the purpose.

We can look into the one issue you pointed out where it should be returning a new response once the caching time expires.

As this doesn't seem to be affecting critical areas (list and personal data), I'm going to lower the priority.

[Michael Johnson]

Closed with 22611e9