I can summarize a pdf in My Drive. Prompt: "Summarize https://drive.google.com/file/d/{{ ID }}/view?usp=sharing" which successfully runs the tools:
people.getMe -> time.getTimeZone -> drive.downloadFile
However, if you copy that file to a shared drive then drive.downloadFile fails with "error": "File not found: {{ ID }} ."