{"id":349,"date":"2024-02-24T22:19:00","date_gmt":"2024-02-24T22:19:00","guid":{"rendered":"https:\/\/ghothi.co.uk\/?p=349"},"modified":"2025-05-28T08:47:47","modified_gmt":"2025-05-28T08:47:47","slug":"automating-my-daily-construction-news-summary","status":"publish","type":"post","link":"https:\/\/ghothi.co.uk\/?p=349","title":{"rendered":"Automating Construction News"},"content":{"rendered":"\n

Reading industry news is part of the job. But doing it manually every day\u2014clicking headlines, skimming paragraphs, filtering out noise\u2014is a time sink.<\/p>\n\n\n\n

So I automated it.<\/p>\n\n\n\n

The script checks sites like ICE, New Civil Engineer, Construction Enquirer etc., pulls the latest articles, summarises them with GPT, and compiles it all into a single clean HTML file. I open one page, skim the key points, and move on.<\/p>\n\n\n\n

The Workflow<\/h2>\n\n\n\n
    \n
  1. Fetch homepage HTML<\/code><\/li>\n\n\n\n
  2. Extract article links<\/code><\/li>\n\n\n\n
  3. Filter by publish date<\/code><\/li>\n\n\n\n
  4. Scrape & clean article body<\/code><\/li>\n\n\n\n
  5. Convert to markdown<\/code><\/li>\n\n\n\n
  6. Summarise with GPT<\/code><\/li>\n\n\n\n
  7. Build HTML report<\/code><\/li>\n<\/ol>\n\n\n\n

    Summaries look like this:<\/p>\n\n\n\n

    <\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
    # Title<\/span><\/span>\n<\/span>\n## Summary<\/span><\/span>\n<\/span>\n\u2022 Key facts in 4\u20135 bullets<\/span><\/span>\n<\/span>\n## Lessons (Optional - only included if there's a clear takeaway for consultants or engineers)<\/span><\/span>\n<\/span>\n\u2022 Key lessons in 2 bullets<\/span><\/span>\n<\/span>\nLink: [URL link to read more]<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n
    Why GPT?<\/h2>\n\n\n\n
    Most news summarisation tools just truncate the article or paraphrase the intro. Most capable LLMs actually understands both the content and context. Because of this, it can be really good at pulling useful insights for technical topics like engineering, infrastructure, regulation, etc provided it’s pulling it from a reference source provided in the prompt, and not just from its own latent space.<\/p>\n\n\n\n
    I’ve purposely written my prompt to ignore fluff and only output clear, concrete points. It also skips the \u201cLessons\u201d section if there\u2019s nothing meaningful to extract.<\/p>\n\n\n\n
    Under the Hood<\/h2>\n\n\n\n
    Article extraction and cleaning<\/h3>\n\n\n\n
    <\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
    def<\/span> <\/span>_extract_title<\/span>(<\/span>soup<\/span>):<\/span><\/span>\n    h1 <\/span>=<\/span> soup<\/span>.<\/span>find<\/span>(<\/span>"<\/span>h1<\/span>"<\/span>,<\/span> <\/span>class_<\/span>=<\/span>"<\/span>name entry-title<\/span>"<\/span>)<\/span><\/span>\n    <\/span>if<\/span> h1<\/span>:<\/span><\/span>\n        <\/span>return<\/span> h1<\/span>.<\/span>get_text<\/span>(<\/span>strip<\/span>=True<\/span>)<\/span><\/span>\n    <\/span>return<\/span> soup<\/span>.<\/span>find<\/span>(<\/span>"<\/span>title<\/span>"<\/span>).<\/span>get_text<\/span>(<\/span>strip<\/span>=True<\/span>)<\/span><\/span>\n<\/span>\ndef<\/span> <\/span>_clean_article<\/span>(<\/span>article<\/span>):<\/span><\/span>\n    <\/span>for<\/span> img <\/span>in<\/span> article<\/span>.<\/span>find_all<\/span>(<\/span>"<\/span>img<\/span>"<\/span>):<\/span> img<\/span>.<\/span>decompose<\/span>()<\/span><\/span>\n    <\/span>for<\/span> a <\/span>in<\/span> article<\/span>.<\/span>find_all<\/span>(<\/span>"<\/span>a<\/span>"<\/span>):<\/span> a<\/span>.<\/span>replace_with<\/span>(<\/span>a<\/span>.<\/span>get_text<\/span>())<\/span><\/span>\n    <\/span>return<\/span> <\/span>str<\/span>(<\/span>article<\/span>)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n
    Strips out irrelevant clutter, like ads, links, images, and just leaves the readable content.<\/p>\n\n\n\n
    Article summarisation<\/h3>\n\n\n\n
    <\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
    _PROMPT_TEMPLATE <\/span>=<\/span> <\/span>"""<\/span>Summarise the article below into 4-5 bullet points. Each bullet should be specific and include only the most important information, without verbosity or fluff.<\/span><\/span>\n<\/span>\nAfter the summary, add 1\u20132 bullet points with practical learning points relevant to a civil engineer working on the pre-construction stage of major infrastructure projects. Only include learning points if there is something concrete to learn.<\/span><\/span>\n<\/span>\nFormat your response as:<\/span><\/span>\n## Summary<\/span><\/span>\n[4-5 bullets]<\/span><\/span>\n## Lessons <only if applicable><\/span><\/span>\n[1-2 bullets]<\/span><\/span>\n<\/span>\n<ARTICLE><\/span><\/span>\n<<ARTICLE_PLACEHOLDER>><\/span><\/span>\n<\/ARTICLE><\/span><\/span>\n"""<\/span><\/span>\n<\/span>\ndef<\/span> <\/span>_summarise<\/span>(<\/span>article_md<\/span>):<\/span><\/span>\n    prompt <\/span>=<\/span> _PROMPT_TEMPLATE<\/span>.<\/span>replace<\/span>(<\/span>"<\/span><<ARTICLE>><\/span>"<\/span>,<\/span> article_md<\/span>)<\/span><\/span>\n    <\/span>return<\/span> client<\/span>.<\/span>ChatCompletion<\/span>.<\/span>create<\/span>(<\/span><\/span>\n                <\/span>model<\/span>=<\/span>"<\/span>gpt-3.5-turbo<\/span>"<\/span>,<\/span><\/span>\n                <\/span>messages<\/span>=<\/span>[<\/span><\/span>\n                    <\/span>{<\/span>"<\/span>role<\/span>"<\/span>:<\/span> <\/span>"<\/span>system<\/span>"<\/span>,<\/span> <\/span>"<\/span>content<\/span>"<\/span>:<\/span> system_role<\/span>},<\/span><\/span>\n                    <\/span>{<\/span>"<\/span>role<\/span>"<\/span>:<\/span> <\/span>"<\/span>user<\/span>"<\/span>,<\/span> <\/span>"<\/span>content<\/span>"<\/span>:<\/span> my_prompt<\/span>},<\/span><\/span>\n                <\/span>],<\/span><\/span>\n                <\/span>temperature<\/span>=<\/span>temp<\/span>,<\/span><\/span>\n            <\/span>)<\/span><\/span>\n    )<\/span>.[<\/span>"<\/span>choices<\/span>"<\/span>][<\/span>0<\/span>][<\/span>"<\/span>message<\/span>"<\/span>][<\/span>"<\/span>content<\/span>"<\/span>]<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n
    Report generation<\/h3>\n\n\n\n
    <\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
    html <\/span>=<\/span> <\/span>f<\/span>"""<!DOCTYPE html><\/span><\/span>\n<html lang="en"><\/span><\/span>\n<head><meta charset="utf-8"\/><title><\/span>{<\/span>page_title<\/span>}<\/span><\/title><\/span><\/span>\n<style><\/span>{<\/span>_PAGE_CSS<\/span>}<\/span><\/style><\/head><\/span><\/span>\n<body><h1><\/span>{<\/span>page_title<\/span>}<\/span><\/h1><\/span>{<\/span>body<\/span>}<\/span><\/span>\n<footer>Report generated on <\/span>{<\/span>timestamp<\/span>}<\/span>.<\/footer><\/body><\/html>"""<\/span><\/span>\n<\/span>\nout_file<\/span>.<\/span>write_text<\/span>(<\/span>html<\/span>,<\/span> <\/span>encoding<\/span>=<\/span>"<\/span>utf-8<\/span>"<\/span>)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n
    No dependencies beyond a few standard Python libraries and the OpenAI client. Runs in ~20 seconds, and the output is saved to a HTML file that I can easily open in my browser.<\/p>\n\n\n\n
    Output Example<\/h2>\n\n\n\n
    Here’s an example output produced from the script with absolutely zero editing from me:<\/p>\n\n\n\n\n\n\n  \n  \n  NCE Report \u2014 22 February 2024<\/title>\n  <style>\n    \/* make padding & border count inside the width *\/\n    *, *::before, *::after { box-sizing: border-box; }\n\n    body {\n      margin: 0;\n      font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;\n      line-height: 1.6;\n    }\n\n    \/* the new wrapper *\/\n    .report-box {\n      max-width: 780px;\n      margin: 1.5rem auto;\n      border: 1px solid #ffffff; \/* 1px black *\/\n      padding: 15px;          \/* 15px inside *\/\n    }\n\n    \/* your original styles below\u2026 *\/\n    .report-box article { margin-bottom: 4rem; }\n\n    .report-box article h2 {\n      color: #ffffff;\n      margin-top: 0;\n    }\n\n    .report-box article ul {\n      margin-top: .4rem;\n    }\n\n    .report-box section.lessons h3 {\n      color: #ffffff;\n      margin-bottom: .25rem;\n    }\n\n    .report-box a.read-more {\n      display: inline-block;\n      margin-top: .5rem;\n      color: #ffffff;\n      text-decoration: none;\n      font-weight: 500;\n    }\n\n    .report-box a.read-more:hover,\n    .report-box a.read-more:focus {\n      text-decoration: underline;\n    }\n\n    .report-box hr {\n      border: none;\n      border-top: 1px solid #ffffff;\n      margin: 2rem 0;\n    }\n\n    .report-box footer {\n      text-align: center;\n      font-size: .875rem;\n      color: #ffffff;\n      margin-top: 3rem;\n    }\n  <\/style>\n<\/head>\n<body>\n\n  <div class=\"report-box\">\n  <h1>NCE Report \u2014 22 February 2024<\/h1>\n\n  <article>\n    <h2>Silvertown Tunnel opening advanced to early 2025 as cut and cover and road realignment progress<\/h2>\n    <h2>Summary<\/h2>\n    <p>\n      \u2022 The Silvertown Tunnel opening date has been advanced to early 2025 (Q4 2024\/25), previously planned for mid-2025\/26, due to programme mitigations.<br\/>\n      \u2022 The project budget has increased by \u00a32.7M to \u00a3176M to cover bus infrastructure, with a current estimated final cost of \u00a3180M due to inflation and land purchase costs.<br\/>\n      \u2022 Significant progress at the Greenwich site includes completed steel framework of the southern portal, dome installation over the cut-and-cover section, and ongoing reinforced concrete works.<br\/>\n      \u2022 At the Silvertown site, steel framework completion, cladding, dome installation, concrete pours, and waterproofing of the tunnelling have been finished, with utilities and highway works underway.<br\/>\n      \u2022 Several planned Blackwall Tunnel weekend closures will enable road realignment, drainage, gantry installation, and resurfacing to support the new tunnel configuration.\n    <\/p>\n    <h2>Lessons<\/h2>\n    <p>\n      \u2022 Innovative construction techniques (e.g., nitrogen skate for TBM rotation, ground freezing for excavation) can accelerate progress on complex underground infrastructure projects.<br\/>\n      \u2022 Effective coordination of temporary traffic management\u2014including planned closures and temporary bridges\u2014is crucial for maintaining traffic flow while enabling major construction activities.\n    <\/p>\n    <p>\n      <a class=\"read-more\" href=\"https:\/\/www.newcivilengineer.com\/latest\/silvertown-tunnel-opening-advanced-to-early-2025-as-cut-and-cover-and-road-realignment-progress-22-02-2024\/\" target=\"_blank\">\n        Read full article \u2192\n      <\/a>\n    <\/p>\n  <\/article>\n  <hr\/>\n\n  <article>\n    <h2>Legal challenge against trio of A47 upgrades dismissed but may go to Supreme Court<\/h2>\n    <h2>Summary<\/h2>\n    <p>\n      \u2022 Legal challenge against three A47 road upgrade schemes in Norfolk was dismissed by the Court of Appeal, with the Development Consent Orders (DCOs) upheld.<br\/>\n      \u2022 Claimant Andrew Boswell argued the Department for Transport and National Highways failed to properly assess the cumulative climate impact of the road projects.<br\/>\n      \u2022 Court ruled cumulative carbon emissions were adequately assessed within current scientific understanding and legal carbon budgets.<br\/>\n      \u2022 Boswell is considering appealing to the Supreme Court, citing concerns over emissions from four major nearby road projects including the Norwich Western Link.<br\/>\n      \u2022 Norfolk County Council and National Highways welcomed the ruling, highlighting the projects\u2019 importance for reducing congestion, improving safety, and supporting regional growth.\n    <\/p>\n    <h2>Lessons<\/h2>\n    <p>\n      \u2022 Infrastructure projects must carefully document and justify environmental impact assessments, especially cumulative climate impacts, to withstand legal scrutiny.<br\/>\n      \u2022 Clear communication of assessment methodologies and adherence to statutory carbon budgets can strengthen approval defenses in infrastructure disputes.\n    <\/p>\n    <p>\n      <a class=\"read-more\" href=\"https:\/\/www.newcivilengineer.com\/latest\/legal-challenge-against-trio-of-a47-upgrades-dismissed-but-may-go-to-supreme-court-22-02-2024\/\" target=\"_blank\">\n        Read full article \u2192\n      <\/a>\n    <\/p>\n  <\/article>\n  <hr\/>\n\n  <article>\n    <h2>Great British Railways will increase data sharing for better digital twins, industry figures say<\/h2>\n    <h2>Summary<\/h2>\n    <p>\n      \u2022 Great British Railways (GBR) aims to unify UK rail management and enhance data sharing across the sector to improve digital twin technology usage.<br\/>\n      \u2022 Digital twins are seen as models representing real-world infrastructure, used for analysis, experimentation, and decision-making, though standards and definitions remain unclear.<br\/>\n      \u2022 Collaboration between different organizations and sectors (e.g., meteorological services and track monitoring) is key to creating richer, more effective digital twins.<br\/>\n      \u2022 Network Rail is trialing a model combining granular weather forecasts with risk predictions for earthworks failures to optimize train speeds on affected lines.<br\/>\n      \u2022 The Rail Reform Bill supports GBR\u2019s goal of breaking down data-sharing barriers to enable more joined-up decision-making and better infrastructure resilience.\n    <\/p>\n    <h2>Lessons<\/h2>\n    <p>\n      \u2022 Effective digital twin implementation on major infrastructure projects requires cross-sector collaboration and open data sharing to bridge gaps in legacy asset information.<br\/>\n      \u2022 Establishing clear collaboration frameworks and government facilitation can help unify fragmented data sources, driving better risk management and operational efficiency.\n    <\/p>\n    <p>\n      <a class=\"read-more\" href=\"https:\/\/www.newcivilengineer.com\/latest\/great-british-railways-will-increase-data-sharing-for-better-digital-twins-industry-figures-say-22-02-2024\/\" target=\"_blank\">\n        Read full article \u2192\n      <\/a>\n    <\/p>\n  <\/article>\n  <hr\/>\n\n  <article>\n    <h2>Sizewell C early works to commence with site creation and ground trials<\/h2>\n    <h2>Summary<\/h2>\n    <p>\n      \u2022 Sizewell C project has triggered its development consent order, enabling early works including site creation, infrastructure improvements (bypass, A12, park and rides), and ground trials to start.<br\/>\n      \u2022 Early construction involves site roads, perimeter fencing, welfare units, ecological and archaeological works, and drainage and earthworks on soft marshy ground.<br\/>\n      \u2022 A 2km long, 50m deep reinforced concrete cut-off diaphragm wall will form a watertight earth-retaining barrier around the nuclear site before excavating about 4 million cubic meters of material without blasting.<br\/>\n      \u2022 The project is funded by EDF and the UK government, which owns 50% and recently committed an additional \u00a31.3bn to support early construction ahead of a final investment decision.<br\/>\n      \u2022 Sizewell C aims to deliver 3.2GW capacity powering 6 million households, supporting the UK\u2019s goal to quadruple nuclear generation by 2050; tunneling works are decoupled from main excavation, unlike Hinkley Point C.\n    <\/p>\n    <h2>Lessons<\/h2>\n    <p>\n      \u2022 Leveraging lessons and extensive ground investigations from previous projects (Hinkley Point C) is crucial for developing cost-effective construction techniques in challenging geology.<br\/>\n      \u2022 Separating tunneling works from main excavation activities can offer improved management and risk control in complex infrastructure projects.\n    <\/p>\n    <p>\n      <a class=\"read-more\" href=\"https:\/\/www.newcivilengineer.com\/latest\/sizewell-c-early-works-to-commence-with-site-creation-and-ground-trials-22-02-2024\/\" target=\"_blank\">\n        Read full article \u2192\n      <\/a>\n    <\/p>\n  <\/article>\n\n  <footer>\n    Report generated on 22 February 2024, 22:52.\n  <\/footer>\n<\/div>\n\n<\/body>\n<\/html>\n\n\n\n<p><\/p>\n\n\n\n<p>It doesn’t really replace reading the full articles, but it definitely keeps me up to date and quickly tells me which article I might be interested in.<\/p>\n\n\n\n<p>The next step would be to schedule it via a cron job so it automatically emails me this each morning…<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Reading industry news is part of the job. But doing it manually every day\u2014clicking headlines, skimming paragraphs, filtering out noise\u2014is a time sink. So I automated it. The script checks sites like ICE, New Civil Engineer, Construction Enquirer etc., pulls the latest articles, summarises them with GPT, and compiles it all into a single clean HTML […]<\/p>\n","protected":false},"author":1,"featured_media":353,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,5],"tags":[],"class_list":["post-349","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml-automation","category-civil-engineering"],"_links":{"self":[{"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/349","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=349"}],"version-history":[{"count":4,"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/349\/revisions"}],"predecessor-version":[{"id":356,"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/349\/revisions\/356"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=\/wp\/v2\/media\/353"}],"wp:attachment":[{"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=349"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=349"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ghothi.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=349"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}