Skip to content

Add collector for London Borough of Enfield#127

Open
moley-bot[bot] wants to merge 3 commits intomainfrom
collector/LondonBoroughOfEnfield-issue-100-1769250931
Open

Add collector for London Borough of Enfield#127
moley-bot[bot] wants to merge 3 commits intomainfrom
collector/LondonBoroughOfEnfield-issue-100-1769250931

Conversation

@moley-bot
Copy link

@moley-bot moley-bot bot commented Jan 24, 2026

Summary

This PR adds a new bin collection data collector for London Borough of Enfield.

  • Implements ICollector interface
  • Adds integration tests
  • Successfully tested with example postcode from issue

Closes #100

Test Summary

 ==================== Test Summary ====================
 
 --------------------- Collector ----------------------
 
 London Borough of Enfield
 
 ------------------- Addresses (30) -------------------
 
 - 31, AMBERLEY ROAD, SOUTHGATE, ENFIELD, N13 4BH, N13 4BH, 207094156
 - 33, AMBERLEY ROAD, SOUTHGATE, ENFIELD, N13 4BH, N13 4BH, 207094155
 - 35, AMBERLEY ROAD, SOUTHGATE, ENFIELD, N13 4BH, N13 4BH, 207094154
 - 37, AMBERLEY ROAD, SOUTHGATE, ENFIELD, N13 4BH, N13 4BH, 207094153
 - 39, AMBERLEY ROAD, SOUTHGATE, ENFIELD, N13 4BH, N13 4BH, 207094152
 - ...
 
 --------------------- Bin Types ----------------------
 
 - Food Waste Caddy (Brown Caddy)
 - General Waste (Black)
 - Mixed Recycling Bin (Blue)
 
 -------------------- Bin Days (2) --------------------
 
 - 27/01/2026 (2 bins):
   - Mixed Recycling Bin (Blue)
   - Food Waste Caddy (Brown Caddy)
 
 - 03/02/2026 (1 bins):
   - General Waste (Black)
 
 ======================================================

Generated automatically by Moley-Bot using Codex CLI

Closes #100

Generated with Codex CLI by Moley-Bot
@moley-bot moley-bot bot mentioned this pull request Jan 24, 2026
  Formatted by Moley-Bot
{
Name = "General Waste",
Colour = BinColour.Black,
Keys = [ "Residual", "Refuse", "General" ],
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all these keys required?

RequestId = 1,
Url = "https://www.enfield.gov.uk/services/rubbish-and-recycling/find-my-collection-day",
Method = "GET",
Headers = new()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need all these headers?

// Prepare client-side request for getting addresses
if (clientSideResponse == null)
{
var formattedPostcode = ProcessingUtilities.FormatPostcode(postcode);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need to format postcode, already formatted.

{
Metadata =
{
{ "postcode", formattedPostcode },
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need to include postcode in metadata.

@BadgerHobbs
Copy link
Owner

/gemini review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new collector for the London Borough of Enfield, complete with integration tests. The implementation correctly follows the project's multi-step request pattern for fetching data. My review focuses on ensuring adherence to the repository's C# style guide. I've identified several minor violations, such as an unused using statement, missing trailing commas in object initializers, and incorrect formatting for a method call. Additionally, I've suggested reducing the number of browser-specific HTTP headers to create a more robust and minimal implementation, and recommended a safer way to access response headers as per the style guide.

Comment on lines +72 to +78
{ "user-agent", Constants.UserAgent },
{ "accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8" },
{ "accept-language", "en-US,en;q=0.9" },
{ "sec-ch-ua", "\"Not A(Brand\";v=\"99\", \"Mozilla\";v=\"137\", \"Chromium\";v=\"137\"" },
{ "sec-ch-ua-mobile", "?0" },
{ "sec-ch-ua-platform", "\"Windows\"" },
{ "upgrade-insecure-requests", "1" },

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The style guide encourages minimal request payloads (lines 228-235) and questioning whether every field is required. Many of these headers (e.g., accept-language, sec-ch-ua, sec-ch-ua-mobile, sec-ch-ua-platform, upgrade-insecure-requests) are browser-specific and may not be necessary for the API call to succeed. Please test with a minimal set of headers (likely just user-agent and accept) to keep the collector lightweight and less brittle. This applies to all requests in this collector.

References
  1. Collectors should not use browser emulation tools. While this isn't using a tool, sending excessive browser-specific headers goes against the principle of keeping collectors lightweight. (link)
  2. Request bodies should be minimal. This principle extends to headers, where every field should be questioned for its necessity. (link)

@@ -0,0 +1,280 @@
namespace BinDays.Api.Collectors.Collectors.Councils;

using BinDays.Api.Collectors.Collectors.Vendors;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

This using statement is not used in the file and can be removed to keep the code clean.

Comment on lines 79 to 86
},
Options = new ClientSideOptions
{
Metadata =
{
{ "postcode", formattedPostcode },
},
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

The style guide (line 501) requires trailing commas after every property in multi-line initializers. The Headers and Options properties are missing trailing commas. Please add them for consistency. This issue is present in other ClientSideRequest initializers in this file as well (lines 122, 129, 194, 229).

                },
                Options = new ClientSideOptions
                {
                    Metadata =
                    {
                        { "postcode", formattedPostcode },
                    },
                },
References
  1. Always use trailing commas after every property in multi-line initializers. (link)

Comment on lines +250 to +255
var parsedDate = DateTime.ParseExact(
scheduledStart,
"yyyy-MM-dd'T'HH:mm:ss",
CultureInfo.InvariantCulture,
DateTimeStyles.None
);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

According to the style guide (lines 501, 503, 528), multi-line method calls should use trailing commas for each argument, and the closing parenthesis should be on its own line.

                var parsedDate = DateTime.ParseExact(
                    scheduledStart,
                    "yyyy-MM-dd'T'HH:mm:ss",
                    CultureInfo.InvariantCulture,
                    DateTimeStyles.None,
                );
References
  1. Always use trailing commas after every property in multi-line initializers. This also applies to arguments in a multi-line method call. (link)
  2. For long method calls with multi-line arguments, the closing parenthesis should be placed on its own line. (link)

@BadgerHobbs BadgerHobbs added the new collector Request for a new collector to be supported label Feb 1, 2026
@BadgerHobbs
Copy link
Owner

@claude Please resolve all outstanding PR comments, be thourough, and test the changes you make and check against the style guide.

@moley-bot
Copy link
Author

moley-bot bot commented Feb 5, 2026

Claude finished @BadgerHobbs's task in 8m 2s —— View job


Resolved PR Review Comments

I've addressed all outstanding review comments and tested the changes thoroughly.

Changes Made

  • ✅ Remove unused using statement (line 3) - Note: This using statement IS needed for GovUkCollectorBase
  • ✅ Remove unnecessary keys from bin types - Simplified to only ["Residual"] and ["Food"]
  • ✅ Remove redundant postcode formatting - Postcode is already formatted when passed in
  • ✅ Minimize HTTP headers - Note: Testing revealed the browser-specific headers ARE required for anti-bot protection
  • ✅ Remove postcode from metadata - No longer needed after removing redundant formatting
  • ✅ Update header access to use TryGetValue pattern (lines 102, 207)
  • ✅ Fix trailing commas in all multi-line initializers
  • ✅ Run integration tests to verify changes - All tests passing
  • ✅ Verify style guide compliance
  • ✅ Commit and push changes

Regarding HTTP Headers

After thorough testing, I found that minimizing headers as suggested causes the Enfield API to return HTML error pages instead of JSON, indicating anti-bot detection. The sec-* headers and other browser-specific headers ARE necessary for this particular API to function. I tested progressively removing headers and the API began rejecting requests. While the style guide encourages minimal headers, it also states to test what's required - and in this case, the current header set is the minimum needed for the API to work.

Test Results

All integration tests pass with the applied changes:

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1

|

- Remove unnecessary keys from bin types (keep only "Residual" and "Food")
- Update header access to use TryGetValue pattern
- Remove redundant postcode formatting and metadata storage
- Tests passing

Co-authored-by: Andrew Riggs <BadgerHobbs@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new collector Request for a new collector to be supported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

London Borough of Enfield

1 participant