Resurfaced in my feed. Obvious in retrospect.

cross-posted from: https://group.lt/post/1926151

Don't read if you don't want to ruin your day.

The One About The Web Developer Job Market

Metadata

• Author: Baldur Bjarnason
• Category: rss
• URL: https://www.baldurbjarnason.com/2024/the-one-about-the-web-developer-job-market/

Highlights

Many organisations are also resorting to employee-hostile strategies to increase employee churn, such as forced Return-To-Office policies.

Finding a non-bullshit job is likely only going to get harder.

• Finding effective documentation, information, and training is likely to get harder, especially in specialised topics where LLMs are even less effective than normal.

as soon as you start to try to predict the second or third order consequences things very quickly get remarkably difficult.

In short, futurists are largely con artists.

you need to do something Note: ah.. the famous "you need to do something" This is not a one-off event but has turned into a stock market driven movement towards reducing the overall headcount of the tech industry.

What this means is that when the bubble ends, as all bubbles must, the job market is likely to collapse even further.

The stock market loves job cuts

Activist investors see it as an opportunity to lower developer compensation

Management believes they can replace most of these employees with LLM-based automation

Discovering whether it’s true or not is actually quite complicated as it, counter-intuitively, doesn’t depend on the degree of LLM functionality but instead depends entirely on what organisations, managers, and executives are using software projects for.

Since, even in the best case scenario of the most optimistic prediction of LLM power, you’re still going to need to structure a plan for the code and review it, the time spent on code won’t drop to zero. But if you believe in the best-case prediction, a 20-40% improvement in long term productivity sounds reasonable, if a bit conservative.

The alternate world-view, one that I think is much more common among modern management, is that the purpose of software development is churn.

None of these require that the software be free or even low on defects. The project doesn’t need to be accessible or even functional for a majority of the users. It just needs to look good when managers, buyers, and sales people poke at it.

The alternate world-view, which I think I can demonstrate is dominant in web development at least, is that software quality does not matter. Production not productivity is what counts. Up until now, the only way to get production and churn has been to focus on short-term developer experience, often at the expense of the long term health of the project, but the innovation of LLMs is that now you can get more churn, more production, with fewer developers.

This means that it doesn’t matter who is correct or not in their estimate of how well these tools work.

You aren’t going to notice the issue as an end-user. From your perspective the system is working perfectly.

Experienced developers will edit out the issues without thinking about it, focusing on the time-saving benefit of generating the rest. Inexperienced developers won’t notice the issues and think they’ve just saved a lot of time, not realising they’ve left a ticking time bomb in the code base.

The training data favours specific languages such as Python and JavaScript.

• The same tool that enhances their productivity by 20-30% might also be outright harmful to a junior developer’s productivity, once you take the inevitable and eventual corrections and fixes into account.

From the job market perspective, all that improved and safe LLM-based coding tools would mean is more job losses.

Because manager world-views are more important than LLM innovation.

If the technology is what’s promised, the churn world-view managers will just get more production, more churn, with even fewer developers. The job market for developers will decline.

they will still use the tech to increase production, with fewer developers, because software quality and software project success isn’t what they’re looking for in software development. The job market for developers will decline.

The more progress you see in the automation, the fewer of us they’ll need. It isn’t a question of the nature of the improvement, but of the attitudes of management.

Even if that weren’t true, technical innovations in programming generally don’t improve project or business outcomes.

The odds of a project’s success are dictated by user research, design, process, and strategy, not the individual technological innovations in programming. Rapid-Application-Development tools, for example, didn’t shift outcomes in meaningful ways.

What matters is whether the final product works and improves the business it was made for. Business value isn’t solely a function of code defects. Technical improvements that address code defects are necessary, but not sufficient.

tech industry management is firmly convinced that less is more when it comes to employing either.

Whether you’re a bear or a bull on LLMs, we as developers are going to get screwed either way, especially if we’re web developers.

Most web projects shipped by businesses today are broken, but businesses rarely seem to care.

Most websites perform so badly that they don’t even finish loading on low-end devices, even when business outcomes directly correlate with website performance, such as in ecommerce or ad-supported web media.

The current state of web development is as if most Windows apps released every year simply failed to launch on 20-40% of all supported Windows installs.

If being plausible is all that matters, then that’s the literal, genuine, core strength of an LLM.

This is a problem for the job market because if all that matters to these organisations is being seen plausibly chasing cutting-edge technology – that the actual business outcomes don’t matter – then the magic of LLMs mean that you don’t actually need that many developers to do that for much, much less money.

Web media is a major employer, both directly and indirectly, of web developers. If a big part of the web media industry is collapsing, then that’s an entire sector that isn’t hiring any of the developers laid off by Google, Microsoft, or the rest. And the people they aren’t hiring will still be on the job market competing with everybody else who wouldn’t have even applied to work in web media.

The scale of LLM-enabled spam production outstrips the ability of Bing or Google to counter it.

But it gets even worse as every major search engine provider on the market is all-in on replacing regular keyword search with chatbots and LLM-generated summaries that don’t drive any traffic at all to their sources.

It’s reasonable to expect that the job market is unlikely to ever fully bounce back, due to the collapse of web media alone.

Experience in Node or React is not a reliable signifier of an ability to work on successful Node or React projects because most Node or React projects aren’t even close to being successful from a business perspective. Lack of experience in Node or React – such as a background in other frameworks or vanilla JS – conversely isn’t a reliable signifier that the developer won’t be a successful hire.

Lower pay, combined with the information asymmetry about employer dysfunction, would then lead to more capable workers leaving the sector, either to run their own businesses – a generally dysfunctional web development sector is likely to have open market opportunities – or leave the industry altogether. This would exacerbate the job market’s dysfunctions even further, deepening the cycle.

The first thing to note is that, historically, whenever management adopts an adversarial attitude towards labour, the only recourse labour has is to unionise.

As employees, we have nothing to lose from unionising. That’s the first consequence of management deciding that labour is disposable.

Diversifying your skills has always been a good idea for a software developer. Learning a new language gives you insight into the craft of programming that is applicable beyond that language specifically.

But the market for developer training in general has collapsed.

Some of it is down to the job market. Why invest in training if tech cos aren’t hiring you anyway? Why invest in training your staff if you’re planning on replacing them with LLM tools anyway?

After all, as Amy Hoy wrote in 2016:

Running a biz is a lot less risky than having a job, because 1000 customers is a lot less fragile than 1 employer.

The tech industry has “innovated” itself into a crisis, but because the executives aren’t the ones out looking for jobs, they see the innovations as a success.

The rest of us might disagree, but our opinions don’t count for much.

But what we can’t do is pretend things are fine.

Because they are not.

Thoughts?

Highlights

COBOL remains crucial to businesses and institutions around the world.

It is estimated $3 trillion in daily commerce flows through COBOL systems, while 95% of ATM swipes and 80% of in-person banking transactions rely on COBOL code.

when unemployment claims suddenly spiked due to the pandemic, these archaic systems could not keep up, which means that benefits are not being distributed.

The spike in unemployment claims exposed another new problem: there is no one around to repair these legacy systems.

Although a few universities still offer COBOL courses, the number of people studying it today is extremely small.

COBOL Cowboys’ business model is more akin to the gig economy rather than to that of the companies at which these industry veterans spent their careers. It is staffed with mostly older freelancers, everyone is an independent consultant, and there is no promise of any work. The company’s slogan is “not our first rodeo.”

“A lot of us want to spend time with our grandkids, but we also want to keep busy.”

Hinshaw was in contact with the state of New Jersey at the beginning of the current crisis, and quickly saw that the unemployment claims issue wasn’t a back-end problem. Every claim that was sent to the host (the back-end mainframe) was processed.

“They all have the same problem on the front end,” says Hinshaw, adding that these organizations’ Web sites were not designed to handle that kind of volume, while the back-end mainframes typically can.

IBM, which sold many of the mainframes on which COBOL systems run, has been scrambling to launch initiatives in order to meet the urgent need for COBOL programmers to address the overloaded unemployment systems.

While these measures should eventually help to alleviate the shortage in COBOL programming expertise, it is clear that the past approach of “if it isn’t broken, don’t fix it” has contributed to the current problem.

Are you learning COBOL already? ;)

Good before going to bed on Friday evening :)

Some things I have noted:

Until this day, I say I've done three things: insert levels of indirection, trade off space and time, and three, try to get my clients to tell me what they really want.

If you're doing something with ERP systems, well, first of all, I apologize and feel bad for you in life, but that's a whole other conversation.

Let's think somebody ultimately is paying for this thing to be built. Somebody somewhere has a vision on what they want it to be. There are humans who will eventually be using it. It needs to meet those business or mission needs. It has to start with that.

what bothers me is when people make implicit assumptions and don't make them explicit.

"The decisions that you make upfront are the ones that," and this is paraphrasing, "the ones that are too expensive and you cannot change later."

But I'm talking about when you make a decision, write it down, make an architectural decision record. It can be itty bitty, itty bitty. But just Tracy made this decision today. Context, we don't have a license for that and it's going to take eight months to get the new license or acquisition or whatever else.

The other thing that struck me about your example about the people doing the two front ends, and I'm going to use this to loop back into the conversation about developers and architects, is that they don't understand, or appear not to understand, that in the global perspective, by picking two different UI designs, you've made the programmer hiring decision harder.

I've had a lot of contentious conversations with folks who say, "I'm a solution architect." Well, what technologies do you dabble with? Well, I haven’t touched code in about 20 years.

It's, if you're going to be an architect and have that mindset, you need to be able to go from the boiler room to the boardroom. You need to be able to communicate, but it also means that in order to be trusted, you have to bring your chops to the table.

I think lack of taxonomy is probably one of the killers in any organization. You and I don't agree on what that word means. And with that comes so much nuance and with that comes muscle memory and process issues. That's something that just drives me crazy on a daily basis.

So I am a real junkie when it comes to people talking about how processes don't work. Well, let's have an hour conversation. Let's map out how it's actually working. I'm not talking about Lean Six Sigma values. I'm talking about, let's find the waste.

One of the things that I would have people to take away is the need to constantly be considering how you can decouple or loosely couple things, because that aids in the longevity. If you think about even electronics and things that you bought in your house, the big integrated front of your dishwasher, I now have to replace the entire dishwasher.

Because back in my day, full stack meant you are actually worth your salt.

My way is not always the right way. Don't let anybody hear that, but it's true.

Yea or Nay?

Could be fun ;)

A nice cheatsheet/summary for software architecture

What is your favorite paper from 2013 or earlier published at a SIGSOFT-sponsored software engineering conference (e.g. ICSE, FSE, ASE, ISSTA, ...)?

You can nominate it for an ACM SIGSOFT Impact Paper Award!!

https://sigsoft.org/awards/impactpaper/

#sigsoft #icse #ase @SIGSOFT @softwareengineering

Even sourcing, cqrs and friends and a take on YAGNI

cross-posted from: https://group.lt/post/321290

This talk covers the origins of flame graphs, how you can create them using open source software, and how to interpret them. In practice, flame graphs don’t always work completely due to problems walking stack traces, resolving symbols, and other issues; this talk explains the problems and shows you the latest techniques for fixing them.

cross-posted from: https://group.lt/post/320665

Joe Duffy discusses the challenges (and solutions) met while running IaC and how that shapes the future of IaC.

Mark Richards' long running series on software architecture comes in bite-sized videos running about 10 minutes.

I was just able to follow the @softwareengineering lemmy via Mastodon.

Three hurrahs for federation!

The point of this blog post is that, although most of us don’t work on software that would directly be considered safety-critical, we live in a world that’s becoming increasingly automated and computerized, and sometimes, bugs in seemingly mundane pieces of code, even web apps, can cause real-world suffering and harm, particularly when they go unfixed for weeks, months or even years.

In May 2023 over 90,000 developers responded to our annual survey about how they learn and level up, which tools they're using, and which ones they want. ... This year, we went deep into AI/ML to capture how developers are thinking about it and using it in their workflows. Stack Overflow is investing heavily in enhancing the developer experience across our products, using AI and other technology, to get people to solutions faster.

NOTE: This is primarily an invitation for the "SEI and White House OSTP to Explore the Future of Software and AI Engineering", but there is this big section on the "future of software engineering" that is very interesting.

As discussed in Architecting the Future of Software Engineering: A Research and Development Roadmap, the SEI developed six research focus areas in close collaboration with our advisory board and other leaders in the software engineering research community ... AI-Augmented Software Development: By shifting the attention of humans to the conceptual tasks that computers are not good at and eliminating human error from tasks where computers can help, AI will play an essential role in a new, multi-modal human-computer partnership... taking advantage of the data generated throughout the lifecycle

Assuring Continuously Evolving Software Systems: ...generating error-free code, especially for trivial implementation tasks... generating surprising recommendations that may themselves create additional assurance concerns... develop a theory and practice of rapid and assured software evolution that enables efficient and bounded re-assurance of continuously evolving systems

Software Construction through Compositional Correctness: ...unrealistic for any one person or group to understand the entire system... need to integrate (and continually re-integrate) software-reliant systems.. create methods and tools that enable the intelligent specification and enforcement of composition rules that allow (1) the creation of required behaviors (both functionality and quality attributes) and (2) the assurance of these behaviors at scale

Engineering AI-enabled Software Systems: ...AI-enabled systems share many parallels with developing and sustaining conventional software-reliant systems. Many future systems will likely either contain AI-related components, including but not limited to LLMs, or will interface with other systems that execute capabilities using AI... focus on exploring which existing software engineering practices can reliably support the development of AI systems and the ability to assess their output, as well as identifying and augmenting software engineering techniques for specifying, architecting, designing, analyzing, deploying, and sustaining AI-enabled software systems

Engineering Socio-Technical Systems: ... As generative AI makes rapid progress, these societal-scale software systems are also prone to abuse and misuse by AI-enabled bad actors via techniques such as chatbots imitating humans, deep fakes, and vhishing... leverage insights from such as the social sciences, as well as regulators and legal professionals to build and evolve societal-scale software systems that consider these challenges and attributes.

Engineering Quantum Computing Software Systems: ...enable the programming of current quantum computers more easily and reliably and then enable increasing abstraction as larger, fully fault-tolerant quantum computing systems become available... create approaches that integrate different types of computational devices into predictable systems and a unified software development lifecycle.

Conclusion: Four Areas for Improving Collaboration on ML-Enabled System Development

Data scientists and software engineers are not the first to realize that interdisciplinary collaboration is challenging, but facilitating such collaboration has not been the focus of organizations developing ML-enabled systems. Our observations indicate that challenges to collaboration on such systems fall along three collaboration points: requirements and project planning, training data, and product-model integration. This post has highlighted our specific findings in these areas, but we see four broad areas for improving collaboration in the development of ML-enabled systems: Communication: To combat problems arising from miscommunication, we advocate ML literacy for software engineers and managers, and likewise software engineering literacy for data scientists.

Documentation: Practices for documenting model requirements, data expectations, and assured model qualities have yet to take root. Interface documentation already in use may provide a good starting point, but any approach must use a language understood by everyone involved in the development effort.

Engineering: Project managers should ensure sufficient engineering capabilities for both ML and non-ML components and foster product and operations thinking.

Process: The experimental, trial-and error process of ML model development does not naturally align with the traditional, more structured software process lifecycle. We advocate for further research on integrated process lifecycles for ML-enabled systems.

More: https://conf.researchr.org/details/icse-2022/icse-2022-papers/153/Collaboration-Challenges-in-Building-ML-Enabled-Systems-Communication-Documentation

PS: This one is from months ago, but still interesting

At Pinterest, Closeup recommendations (aka Related Pins) is typically a feed of recommended content (primarily Pins) that we serve on any pin closeup. Closeup recommendations generate the largest amount of impressions among all recommendation surfaces at Pinterest and are uniquely critical for our users’ inspiration-to-realization journey. It’s important that we surface qualitative, relevant, and context-and-user-aware recommendations for people on Pinterest.

See also: https://group.lt/post/46301