-
• #1202
It might need you to specify the
Accept: application/json
header to tell the server you definitely want the response in JSON.Vaguely related: https://techforum.tfl.gov.uk/t/json-vs-xml-random-content-type-in-response/2238/13
-
• #1203
Can you try (based on https://stackoverflow.com/questions/8685790/adding-headers-to-requests-module):-
import requests headers = {'Accept': 'application/json'} url = 'https://data.bus-data.dft.gov.uk/api/v1/datafeed/1695/?api_key=***' response = requests.get(url, headers=headers) ...
-
• #1204
Reading your link "gtfs" might be better to use?
I think GTFS is a specification the transport companies should stick to when publishing this data, not a different way of accessing the data, if that makes sense?
So when the initial doc page said "Results are returned as JSON.", it was wrong. Really, the data adheres to GTFS and is therefore going to be .xml. Unless maybe you force it to be json like greenbank is saying and I was garbling through on the last page -
• #1205
Not sure where I'd stick that line into the code? It sounds like it isn't alternating though, is just always xml.
Thanks though I'm learning some things this evening just not along the path I'd hoped to progress on.
-
• #1206
Ah, I'll try this ta.
-
• #1207
The two lines I've given just replace the existing
requests.get
line. [EDIT] Copied the code from the wrong example so I've updated it now.The link to the alternating problem was to highlight that it "should" change its behaviour based on the value of the
Accept
header, albeit with some bug in the server's implementation.With no header provided (as your existing code does) it may default to XML.
It may appear as JSON when viewed in the browser because the browser is providing the accept header set with it set to (amongst other things)
application/json
. -
• #1208
I think that makes sense,although a quick m Google gives info on gtfs -> json conversion?
It's a whole new day of learning I think and I can usually focus on this only once every few months at best 🤔
-
• #1209
Ok the type is now
text/xml; charset=utf-8
Which is different
Status code was 406 which is a fail. Progress though maybe?
-
• #1210
406 is "not available in preferred format"
-
• #1211
I think xmltodict is the solution, again my thought process & application hampered by examples being local file based with maybe just 1 example data row and only a few headings which my pea-brain can't quickly adapt to what I'd like to do. Running before I can walk, I know.
-
• #1212
Thanks both @yoshy & @Greenbank, it has been useful / humbling / exasperating / humiliating in equal measure for me, you've both been very helpful on the subject since I posted.
Hope you got your budget done too @yoshy
-
• #1213
I found more info, xmltodict does a conversion but I need to get my ahead around keys now it seems? I can get it to show the top level but no further, looks like 7 or 8 levels via the url.
import requests import xmltodict url = 'https://data.bus-data.dft.gov.uk/api/v1/datafeed/1695/?api_key=***' response = requests.get(url) r = xmltodict.parse(response.content) [#print](https://www.lfgss.com/search/?q=%23print)(r) [#useful](https://www.lfgss.com/search/?q=%23useful) check line, showed a more expected json like output print("Status code:", response.status_code) print(response.headers['Content-Type']) result = list(r.keys()) print(result)
Gives no error:
Status code: 200 text/xml ['Siri']
I've no clue how to get it so I can filter by bus service number & direction from here, but that's the plan. Step 1 will be for those fields to show on or after line 3 of the output I guess.
-
• #1214
Do a bit of reading about dictionaries, the general idea being that you have key+value pairs.
If you tell the program:
print(r['Siri'])
You're saying "get me the value from the datastructure called 'r' which is associated with the key 'Siri'"
Similar to if you have an array called arr and want the first value you'll say arr[0].To get further into the file, try adding another set of square brackets:
print(r['Siri'] ['ServiceDelivery'])This might work or I could be making an arse of myself, let's see!
-
• #1215
Thanks, I will read up. C&P of
print(r['Siri'])
Gives the full output (all live busses) so it's picking something up.
print(r['Siri'] ['ServiceDelivery'])
Gives the same, basically every live bus.
If I can get a filter in place I can work with a single LineRef (bus number) which means much less info coming back, that's where reading about dictionaries could/should help.
-
• #1216
So I was just using the screenshot you posted last page to start indexing into the data, using the indents in the file to determine the 'level' of indexing needed. The next level looks like it'll be 'VehicleMonitoringDelivery', within which there are at least 2 'VehicleActivity'.
This is where things get a bit more involved... a key in a dictionary has to be unique, so the xmltodict library is going to have to do some jiggery pokery to make it work (put the VehicleActivity elements in an array: https://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html (I think!))Thanks for posting about this btw, it's inspired me to get my dev environment set up on my pc and get noodling again.
-
• #1217
Also please do tell me to shut up if you just want to work away and suss things out on your own
-
• #1218
the xmltodict library is going to have to do some jiggery pokery to make it work
I bet £5 there will be a package to do this for you...
-
• #1219
The very first example in the
xmltodict
docs page has two<many>
keys and they're concatenated into a list. So hopefully thoseVehicleActivity
items will be fine. -
• #1220
Yeah sorry that's what I meant about the array in my post, my bad
-
• #1221
I'm well out of my depth already, so guidance that helps me understand more and stay afloat or get closer to be able to figuring things out in future is useful thanks.
-
• #1222
Anyone got any experience with NAS cases for a 3b+ beyond the Argon EON?
Is it even worth it without USB 3.0 bus? Which it seems like all Pi based NASs use.
-
• #1223
I put circuit python on a RPi Pico, and then using the HID library I got it to pretend to be a mouse when plugged into a computer.
But when it's plugged into a computer, it also appears as a mass storage device (that's how you update the python code).
So how is it appearing at two devices? I assumed it was setting up a virtual USB hub and "attaching" two devices to it, but that doesn't seem the be the case. When I run
lsusb
I only see one device. I'm so confused -
• #1224
Did a little bit of googling and I'm not claiming this to be definite but I think the one device is opening up two communication channels with the PC over the 1 USB connection. 1 of those comms channels is just having fairly simple generic USB HID messages sent over it and the pc is accepting these and saying "okay there's a mouse there" and on the other comms channels it's sent whatever it needs to to say "this is a generic storage device, you can mount this and use it".
If you run "ls /dev" without it plugged in, then again with it plugged in, how many more devices does it show? Out of interest -
• #1225
Oh well, my twitter timelapse project dies on 9th Feb, along with many other cooler & more useful accounts no doubt
Ah right, t'other page said json for sure, and looking again gives example.
Reading your link "gtfs" might be better to use? Fuck knows now, feels like I'm starting all over again.