- changed status to open
APAScraper
... does no longer work, see http://psycnet.apa.org/record/1996-02773-003
This is a really tricky case: their web page is loaded completely dynamic, that is, via JavaScript. Pressing the export button causes some JSON requests in the background and then finally a file to pop up. Unfortunately, as it is opened in a new window, the normal network trace of Firefox (F12) does not allow us to see what's going on in the background. So this requires some reverse engineering to understand which requests are required. Really sad. :-(
Comments (6)
-
-
- removed responsible
-
getting the doi and scraping that instead should work: https://scraper.bibsonomy.org/service?url=https%3A%2F%2Fdoi.org%2F10.1037%2F0033-2909.119.2.254&selection=+
oh and if you play around with curl you’ll get banned very quicklythis combination helps for debugging:
https://stackoverflow.com/questions/12212504/automatically-open-chrome-developer-tools-when-new-tab-new-window-is-opened
https://stackoverflow.com/questions/23668827/how-to-open-the-chrome-developer-tools-in-a-new-window -
-
assigned issue to
-
assigned issue to
-
Should be fixed in simple-scraper-fixes branch. You only need cookies from the page itself and a part, uid, from the url. Then you have to send a post-request with a json containing the uid to “
https://psycnet.apa.org/api/request/record.exportRISFile”
to get authorization and then a get-method to “https://psycnet.apa.org/ris/download”
to get the RIS, which can be converted to bibtex.
-
- changed status to resolved
- Log in to comment