Julia M. Rohrer @dingdingpeng.the100.ci

Or at least a single page from which you can download all 200 files, rather than opening 200 individual other pages 🥲

3 replies 0 reposts 4 likes

Unfork Statistics @btjo.bsky.social [ View ] Jul 02, 2024 at 13:02 UTC

I've seen R packages which can do this sort of job for you. But I've never used them so can't help beyond that. Not sure what keywords would work apart from being in the same general domain as 'scraping'.

0 replies 0 reposts 1 likes

Bryan Andregg @bandregg.bsky.social [ View ] Jul 02, 2024 at 14:16 UTC

Is there a higher-level page that eventually lead to the 200 other pages even through individual pages for each? That’s a job for curl(1).

0 replies 0 reposts 1 likes

Andrew Mercer @awmercer.bsky.social [ View ] Jul 02, 2024 at 12:53 UTC

I definitely sympathize. The truth is that 98% of the use for web panels is cross-sectional. The panel design is mostly about reducing data collection costs. Everyone (me included) gets very excited about the longitudinal possibilities, but it’s much harder to pull off than anyone expects.

1 replies 0 reposts 3 likes