For many decades, we have known how to write pretty secure software. It takes a bit longer, and security considerations must be central to early design efforts, but it is possible. However, developers have usually been in too much of a rush to market to do this. So most software systems today are riddled with security holes. What has saved them so far is that it takes humans a lot of work to find and exploit such holes.However, there now exist powerful AI systems that are far better at finding and using such holes. Soon (within a year or two?) many AI firms will have such tools, and they will spread to be widely available. Yes, such AI systems can also work to patch such holes, but computer security experts tell me that the nature of insecure systems is to make it much easier to find and use than to patch such holes. Attack beats defense.Software firms would then more eagerly rewrite their code to use more secure designs, and AI could help them to do this. But this takes time, and as there isn’t a lot of secure software out there now, AI hasn’t had big datasets ready to help them learn how to do this well. So it will take some time to replace weak with strong software.
>>62199367So there may soon be a period, starting within a few years, maybe lasting a few years, when most actual software systems can cheaply be hacked. This will make such software firms vulnerable to ransomware, and make customers wary of using their products. Customers, firms, and App stores, will respond by cutting back on what software systems they offer, and by simplifying them by dropping many features.As our world has come to rely on software for a great many things, it seems quite concerning that we might soon have to make do with substantially less software. How vulnerable are crucial systems like electricity, cars, traffic lights, voting systems, and payment systems? I don’t think we know. Beware the coming Hackastrophe.
the low hanging fruit ai finds will disappear. software becomes more secure and stable. hackers don't have the resources to churn out exploits to the degree the corpos have to close the holes
Hackers, even state sponsored, have already been pointing it at open source codebases for a couple of years and I haven’t seen constant headlines of novel attacks they find. I’m suspicious that anthropic has a model that has leaped some kind of divide that churns out vulns like it’s cake
>>62199400Suspicious as in it is fud
