kyrra 18 hours ago

From the source: https://cloud.google.com/blog/topics/threat-intelligence/voi...

> The instance was used to store contact information and related notes for small and medium businesses. Analysis revealed that data was retrieved by the threat actor during a small window of time before the access was cut off. The data retrieved by the threat actor was confined to basic and largely publicly available business information, such as business names and contact details.

  • jedc 16 hours ago

    "store contact information and related notes for small and medium businesses"

    Most likely translation: it affected the Google SMB sales team's Salesforce instance

    • angmarsbane 15 hours ago

      My understanding is that the Cloud org uses Salesforce, the rest of Google uses a self-developed solution.

  • sugarpimpdorsey 16 hours ago

    > Analysis revealed that data was retrieved by the threat actor during a small window of time before the access was cut off.

    That's a pretty nonchalant way to say "they totally stole stuff before we knew what was going on or could stop them".

    • trhway 16 hours ago

      On the other side, giving how slow and cumbersome data extraction from enterprise software, may be they are saying that the hackers also didn't get that much and far.

  • lesuorac 17 hours ago

    > The data retrieved by the threat actor was confined to basic and largely publicly available business information

    Which is to say, they took public _and_ private data and the private data is something we don't wish to publicly admit so probably not good.

  • nitwit005 16 hours ago

    This is generally what people try to steal out of Salesforce. I doubt it's as innocuous as that makes it sound, as they wouldn't bother if they couldn't make money off of it. I assume there is some secondary scheme, like fraudulent billing.

    • coredog64 15 hours ago

      Having seen the AWS version of this type of data store, it's typically got information like billing account numbers, internal email addresses of stakeholders, customer notes about NDA'd strategy, and lists of bugs/feature requests the customer is interested.

      Could totally see someone sending a message like "Hey, your TAM asked me to talk to you about $IMPORTANT_FEATURE_REQUEST, can you grant me read access in the account where you're developing $UPCOMING_SECRET_PROJECT so I can get some additional color?" It might even be enough to get someone on a conference call and pump them for MNPI about $UPCOMING_SECRET_PROJECT under the guise of ensuring that the feature request is helpful.

  • sunaookami 5 hours ago

    I despise communication like this: "it doesn't really matter, it was just a very very very small portion of users with uninteresting data, really, believe us!". Is it some kind of legal thing? Does an actual apology open them up for lawsuits or what?

Bluescreenbuddy 19 hours ago

Surprised Google didn't have some internally developed alternative.

  • progbits 19 hours ago

    From my experience with sales/PM people at google, they refuse to use internal tools and try to get Jira and other shit installed. Regardless of the tool quality, just because that's what they learned already.

    This mostly didn't work out for them back in the day but in more recent times as more and more low quality middle level managers and execs get hired they manage to get approvals.

    In my org a new VP demanded Jira instance within a month of joining the company and that it be used for technical project reporting.

    Of course all the developers said fuck no to that so for a while some managers were trying to do two way sync between Jira and Buganizer. When I left it was mostly abandoned and full of tumbleweed...

    • kwanbix 18 hours ago

      Jira's raise to power is one of those things I would never understand. Such a horribly designed tool. Today is much better, yes, but it is so over-engineer and at the same time lacks so many things.

      • Agingcoder 17 hours ago

        The first time I used it around 2007 I thought it was great. It was basic, but did everything that I wanted ( I’d didn’t care about the project management that maybe didn’t even exist back then I don’t remember ).

        I think that it’s been diverted from its original purpose,and is now indeed horribly complicated since it’s supposed to be all in one package.

        I’ve also noted that in large companies the quality of the product for end users, as long as it’s not a massive drag on productivity or on recruitment and is not core business, is irrelevant and that other factors are more important ( costs, contracts , easy to install integrate and maintain, quality of support, breadth of use within the company etc ). This makes atlassian a natural superpower.

        • mbreese 17 hours ago

          Early 2000s Jira was great because it wasn’t Bugzilla. Bugzilla was functional, but that was about it.

          • SoftTalker 17 hours ago

            Bugzilla was great....

            • mbreese 11 hours ago

              Unless you were a non-technical person — then it was a confusing mess. I think this is part of why Jira did so well, it was more approachable to non-devs.

      • asah 17 hours ago

        Jira was the first tool to truly support bulk search/edit of issues, i.e. it scaled where everything else fell over with >250 issues...

      • lenkite 16 hours ago

        Jira may be over-engineered, but I don't think it lacks anything. You can always get a plugin if something is missing. Our corpo Jira crawled because of a stupendous amount of plugins (close to a thousand). Once we had a Jira clean-up operation done, it became magically fast.

        • npongratz 16 hours ago

          > You can always get a plugin if something is missing.

          To my great consternation, I have not found this to be true in the cloud version:

          https://jira.atlassian.com/browse/JRACLOUD-72631

          Special thanks to Matt Lachman for keeping up the good fight every (business) day.

          • lenkite 15 hours ago

            Huh - that seems a very basic missing feature in the cloud version. We use bog-standard self-hosted JIRA and markdown editing is basic working functionality. People also add mermaid diagrams/charts to the issue. As well as custom diagram plugins, excel sheets and a whole gamut of documents.

        • kbelder 15 hours ago

          Plugins can fix every problem, except the problem of too many plugins.

      • 0xbadcafebee 17 hours ago

        Replace Jira with Microsoft and this is the same complaint from the 90's/2000's about a business company that delivers features rather than making nerds happy. Nobody likes it, yet everybody uses it.

      • crinkly 17 hours ago

        PM like it because they can break it until it fits their worldview. I've worked at 3 orgs in a row where the JIRA was a complete fucking broken mess because the process in it didn't match reality but someone thought it did.

        • bombcar 17 hours ago

          This is exactly it - it's "Enterprise" so you can (pretty easily, to be honest) make it fit your workflow.

          The problem is that the workflow you officially have and wish you used is almost never the actual workflow, so it becomes horribly confused and insane.

        • Terr_ 16 hours ago

          Yeah, my cynical experience with B2B business software is that it becomes shitty and encrusted via special-case customizations (or worse, customizability.)

          Even for internal projects, a lot of money is thrown at software because the corporation has decided (rightly or wrongly) that it's easier than changing process, culture, personnel, or internal incentives.

          For example, salespeople on commission were closing not-very-profitable deals. The response was to layer in a complicated project feasibility/profitability estimation logic, configuration features for an "approval" org-chart hierarchy between users, and various new triggers to block the workflow at particular steps and e-mail people to come click and approval button... I still feel it would have (should have?) been better to change how the sales commissions worked.

      • grumple 17 hours ago

        I think it's fundamentally easy to use once you get it set up, it's just absolutely madness in terms of configuration. But you can easily manage a backlog, sprints, update tickets, etc, plus they have a query language (JQL) that you can use to make widgets that are useful (although many of those should just be defaults). It's got a lot of flexibility in terms of required fields, forms, workflows, etc.

      • infamouscow 18 hours ago

        It's very easy to understand, developers just refuse to accept it for undermining their strongly held beliefs regarding success in the software industry.

        It's true you need working software, but without sales and operations doing their part, the software will be scraped when the company folds.

        Sales and operations get away with everything because they're the beating heart of any successful organization.

        • dilyevsky 17 hours ago

          They also know how to pull strings and engineers generally don't

        • bayindirh 17 hours ago

          Yes, Jira is powerful, flexible and allows tons of stuff to be done. It can really store tons of data, accept workflows, etc.

          But that thing is slow as a snail. Even if it's an on-prem installation. I want nimble tools.

          I know it's a very unpopular opinion, but I'll take a fast Redmine over a slow Jira all day, every day.

          P.S.: Another slow tool like this is OpenStack. Every CLI command, every web UI click means a ping-pong of 20 REST requests. At least, when it works, it works, which is 100% of the time if it's configured correctly.

    • sciurus 17 hours ago

      I saw a similar pattern when I worked at Mozilla. We had bugzilla and jira, mediawiki and confluence, irc/matrix and slack, the list goes on...

      I just checked and https://github.com/mozilla/jira-bugzilla-integration is alive and well.

      • hnlmorg 16 hours ago

        That doesn’t sound that egregious in my opinion.

        Bugzilla is a Mozilla product so you’d hope they’d use it themselves (it’s often referred to as “dogfooding”). But Jira is everywhere so I’m sure some project managers argued that it was needed.

        And once you have Jira then the same people push for Confluence too. But MediaWiki was the de facto standard before everyone jumped on proprietary solutions like Confluence and Notion. In fact I seem to recall that very early versions of Confluence was just a 3rd party Wiki that Atlassian bought. Or at least there was a Java-based Wiki in their early portfolio.

        You also have to bear in mind that organising docs is an endless and thankless job which nobody wants to do. So these things tend to multiply like vermin once someone starts creating docs on another platform. One startup I worked for somehow managed to have stuff scattered between Confluence, Notion and Google Docs despite only employing 50 people. It was crazy.

        Another client I recently worked for had Sharepoint, Notion and Confluence as their official tools for documentation.

        As for IRC and Slack, every company I’ve worked at in the last 5 years had two of either MS Teams, Zoom or Slack. Literally every company. And that’s in addition to email. Go back further and there was Skype, WebEx, and so on and so forth too.

        It’s almost a meme these days to hear the sentence “how would you prefer to be contacted” because so many solutions are competing against each other with overlapping functionality.

        Then you have developer-focused tools like GitHub with their own docs and issue tracking too

        At this point in time, it’s easier to just accept that each org is going to end up with multiple overlapping solutions because you’ll get new people join the team and they’ll want to use their preferred tool because that’s what they’re productive in and so the spiral continues.

        So if Mozilla managed to keep the options down to just 2 for each product category, then I’d say they were doing better than most other organisations.

        • mook 14 hours ago

          Bugzilla isn't so much a Mozilla product as something that was home grown at Netscape because there wasn't much else at the time, and they just kept using due to inertia. Though as a developer I'd still prefer that over Jira, but that's probably because I don't really need any reporting functionality.

          • QuercusMax 14 hours ago

            I've used (and customized) Bugzilla, used Google Buganizer extensively, used Jira for a year and a half, and also built an internal system consisting of a bugtracker + requirements manager + sprint planner + customer management system + manual test tracking tool + knowledge base.

            Bugzilla was fine to hack a few extra fields into, but I wouldn't want to build anything around it. Buganizer was actually pretty nice, but suffered from too many competing tools built around it, most of which were just somebody's 20% project, so they kept getting abandoned. Jira wouldn't be so bad if it weren't so slow and annoying to use; only our TPM can keep track of how everything is set up.

            The internal system I built was very specialized to our use-cases; it started out as a simple task list and eventually grew into a huge beast. By far the worst part of the system was the manual-test-management system, but that was just a mess due to its very nature. We were able to be very efficient with some of the custom functionality we made.

          • hnlmorg 13 hours ago

            They also keep maintaining it too.

            But you’re right, calling it a “product” does somewhat oversell the significance of the project within Mozilla.

    • lenerdenator 19 hours ago

      > From my experience with sales/PM people at google, they refuse to use internal tools and try to get Jira and other shit installed. Regardless of the tool quality, just because that's what they learned already.

      That's when you're supposed to pull the smooth-talking people that are usually in those roles and ask them a very simple question:

      "Do you want this tool more than you want to be employed?"

      • geodel 16 hours ago

        I think software developer's high pay and relatively consequence free existence have given them a bit of thought leader quality in domains beyond their expertise. But it is not going to be the case for lot of developers soon. So pulling things like

        > "Do you want this tool more than you want to be employed?"

        will be harmful to wellbeing of developers rather than sales guys.

        • lenerdenator 16 hours ago

          > I think software developer's high pay and relatively consequence free existence have given them a bit of thought leader quality in domains beyond their expertise.

          Just wait until you hear what salespeople get up to and what they make off of it.

      • closewith 18 hours ago

        Good software salespeople are much rarer than good developers, so it's likely that conversion would be had with the other parties.

        • lenerdenator 16 hours ago

          From what I have experienced, "good" software salespeople are the ones telling clients lies to seal the deal, that then fall back on the software engineers to fulfill in unreasonable amounts of time that compromise the entire project. I wouldn't call the ability to lie a rare trait.

          • closewith 15 hours ago

            From a management perspective, that sounds like:

            * Inspiring client confidence and enthusiasm in our solutions

            * Motivating engineering teams to tackle ambitious challenges

            * Delivering high-impact results within accelerated timelines

            Maybe if the devs hadn't been slacking beforehand, they wouldn't have had to rush to catch up.

            • lenerdenator 14 hours ago

              But you're using management perspective, and we know that's flawed.

              • closewith 14 hours ago

                From a customer perspective, that sounds like:

                * Your sales team passionately championing solutions tailored to my needs

                * Them securing the resources and commitments needed to accelerate delivery

                * Them inspiring the engineering team to rise to ambitious deadlines, ensuring my project stays on track and delivers real value

                Maybe if the devs shared their dedication to meeting my goals head-on, they'd be able to ensure my business objectives would be achieved without having to crunch.

                • ramses0 11 hours ago

                  I'd like to subscribe to your newsletter (unfortunately...)

        • sigmoid10 18 hours ago

          Only if you apply a lower standard for "good" software developers.

          • closewith 17 hours ago

            You could limit your definition of good to the 99th percentile and it would still be true.

            • sigmoid10 15 hours ago

              I'm t might seem so if you only ever worked in tech, but there are huge companies out there that employ way more sales people than technical people.

              • closewith 15 hours ago

                Yes, of course, because all industries need sales and salespeople are extremely valuable to the business.

                But in software, like all industries, the best salespeople are also domain experts, and domain experts in software are rare before you add the need to be able to sell.

    • 8n4vidtmkvmk 15 hours ago

      I find this amusing. I have my own preferences too, but I wouldn't dare even suggest an alternative at a trillion dollar/100k+ employee company. Perhaps because I'm fully aware of what a colossal nightmare that would be. There's enormous value in just sticking with what everyone is already using, even if it's objectively worse. There is a breaking point of course, but the more people that are involved.. that other thing better provide tremendous value and you better be prepared to do a lot of convincing.

    • Wonnk13 16 hours ago

      Man I miss Buganizer... Even in 2017 Google was starting to smell like Accenture and Oracle. Glad i left before Jira was shoved down my throat.

      • surajrmal 7 hours ago

        Buganizer is still the standard and better than ever. There is little reason to try and reach for jira these days.

  • Rebelgecko 18 hours ago

    Google has been replacing a lot of internal tools with janky cookie cutter Salesforce stuff. Part of the culture change I guess.

    • QuercusMax 17 hours ago

      My experience was that a lot of internal tools were tremendously janky. The awful system used for filling out compliance questionnaires for audit often had 10+ second UI latency when saving text fields. The perf tools often broke right when everyone had to use them all at the same time.

      I don't know if they every built a proper replacement, but for at least half a decade the Baggins Roster UI (internal backend for things like Google Groups and such) appeared to have been an abandoned summer intern project.

  • paxys 17 hours ago

    Custom internal tools at such companies are mostly restricted to the engineering org. Employees in sales, marketing, accounting etc. prefer to stick with the industry standard.

  • johannes1234321 19 hours ago

    As long as they don't aim to make it a product developing a CRM is too expensive. Especially if one wants to include country specific requirements etc. Also training users on a custom software costs money and many people working in roles requiring CRM usage rotate relatively fast.

    And for making it a product: It's a quite competed market, with Salesforce, SAP, Google, Microsoft, ... and it doesn't fit to Google's "you're on your own" approach, but requires consulting and integration services, as introducing a CRM to a company involves analysing the existing processes and then adapting processes to software capabilities and adapting software to processes. (Which both often fails ...)

  • matchagaucho 16 hours ago

    A few board members have Google/Salesforce connections. They partner on a lot of tech and markets.

ok123456 18 hours ago
  • superfrank 16 hours ago

    > In June, Google warned that a threat actor they classify as 'UNC6040' is targeting companies' employees in voice phishing (vishing) social engineering attacks to breach Salesforce instances and download customer data

    > [...]

    > In June, one of Google's corporate Salesforce instances was impacted by similar UNC6040 activity described in this post

    Nope. Good old fashion social engineering.

wferrell 19 hours ago

They had an internal CRM. It was buggy, missing key features and engineers didn’t really want to work on it.

  • hnthrow90348765 18 hours ago

    If I had jumped through Google's hiring hoops, I wouldn't either. Of course, this could be solved with money.

    • wferrell 13 hours ago

      I think the real reason was there was no path to promotion for working on this. For better or worse the incentives were not aligned for great work to happen.

mrweasel 18 hours ago

Oh, so I wonder if that's also how KLM lost my data.

shadowgovt 19 hours ago

I'm modestly surprised to learn Google was using Salesforce internally at all; the NIH runs deep with that company (they even have their own bugtracker because every other option just wouldn't cut it).

On the other hand, the past decade-ish has seen them grow very rapidly via acquisition, so perhaps this DB was grandfathered in via an acquired company and hadn't yet been replaced by anything internal.

(For Salesforce in particular though, I'd be willing to believe Google doesn't have an in-house alternative... People asked for a Salesforce-like in Google Workspace for years and the company had no interest. I have a hunch that most Googlers find the idea of creating a new CRM to be a profoundly boring intellectual exercise).

  • eitally 19 hours ago

    Fwiw, I was hired by Google in 2015 to help answer questions like "if Google were to add a CRM to the GSuite portfolio, should they build one, buy one or partner with key players". My team's charter was to create business cases with various options and run them up to chain (at the time, Prabhakar was running product for "Google for Work"). On more than one occasion we presented cases with 3 year ROIs in the $xxxM range and were shot down every time with a "too small" comment. A couple years later, Google had partnered with Copper CRM and supported extension builds into Workspace/GSuite, but had also begun a major enterprise rationalization project to consolidate a multitude of Salesforce instances into a single one, at the same time as adopting standard enterprise features & processes of Anaplan.

    This led to consolidation of a number of back office IT teams that ultimately ended up with far more enforcement clout than they'd historically had. By the time Ruth changed roles, most of the "normal" business processes had been fairly standardized. Fwiw, the Cloud instance of SFDC, which is by far the most complex & customized, has been in full use for almost five years now and is the canonical source of truth for sales data.

    • coredog64 19 hours ago

      I'm surprised Google could get away with only a single SFDC instance. AWS has multiple SFDC installations and is forever having to deal with "Oh, yeah, that data is in this other SFDC installation"

      • ssk42 15 hours ago

        Yeah, they have the world class Salesforce engineers there. One of Google's Salesforce's last tech leads wound up becoming the Director of the proprietary Salesforce language Apex.

    • shadowgovt 18 hours ago

      I wonder if the Cloud SFDC is the one that was compromised. It's a little telling Google didn't go into details about which arm of the octopus got attacked (or if they did, I didn't see that reporting yet... Unless Cloud is the implied victim because the description of the attack showed up on the Cloud blog).

      I feel you about the ROI. In hindsight, it's a little funny to me that Salesforce is doing revenue numbers a little under half of Google Cloud; you'd think that would be large enough value to get Google interested in biting into that pie.

  • loeg 18 hours ago

    > they even have their own bugtracker because every other option just wouldn't cut it

    Of all the things to NIH, this is one of the most defensible -- lots of bugtracker options just aren't very good.

    • 8n4vidtmkvmk 15 hours ago

      I found this to be true too, but I don't really get it. Doesn't seem like that complicated of a software. Maybe I'm only thinking like a SWE, and not PM and other laypersons that also need access.

    • cjpearson 17 hours ago

      I've generally not had an interest in working for one of the big tech companies, but the opportunity to escape JIRA is tempting.

    • kevincox 13 hours ago

      Yeah, Google's Buganizer was the best bug tracker that I ever used.

      ...and it still wasn't great.

      • surajrmal 6 hours ago

        It's pretty much perfect in my eyes. Not being open source is probably the biggest thing I'd fault it for. The world deserves better than GitHub issues and jira, pity it can't be used by anyone else.

  • dilyevsky 19 hours ago

    iirc google cloud’s entire support ticket system is built on top of sf - it went down when saleforce had an outage a few years back

  • bpodgursky 19 hours ago

    Salespeople are VERY familiar with Salesforce and are not very technical. Probably significantly increases onboarding and training time to have a weird new tool.

    Easy to hire experienced salespeople and have them hit the ground fast if they use standard Salesforce conversion flows.

    • bombcar 16 hours ago

      It still amazes me that Salesforce, which is good, mind you, is still basically just Microsoft Access as a Service, and yet here we are.

  • mc32 19 hours ago

    Google uses lots of non-Google solutions for many things —just imagine all the facilities stuff. But so does any software company, including Microsoft and Amazon.

    That said, you can hire people for any purpose (specific roles) and you can build what you want. It’s more a question of whether it’s worth it to build such solutions, after all you have a main line of business to tend to. That’s to say even Google and Apple have so called “boring “ roles and there are lots of people who don’t see it that way and want to work doing those things.

    • progbits 19 hours ago

      Actually lot of the facilities stuff is inhouse too - floor plans (not just the seat map but actual floor drawings that include physical infrastructure); the ticketing system for maintenance; work hour tracking for contractors; probably lot more that I'm forgetting.

      But yes your point stands, sometimes it just makes more sense to use an existing product.

      • eitally 19 hours ago

        The floor plan tool isn't really in house. It's just an extension of the industry standard real estate management platform they use Tririga (https://www.ibm.com/products/tririga) ... in the same way that go/teams in just an custom visualization of a standard employee directory.

        You might be surprised how much of what runs Google (Anaplan, for example, for XWS) is fairly industry standard.

        • scottyah 14 hours ago

          They did acquire (then sell) SketchUp which is what I use for floorplans.

    • shadowgovt 19 hours ago

      Given the low expected profit margin, a CRM solution at Google would likely come from a 20% project (or rather, the equivalent thing these days since last I checked 20% is basically dead as a formal concept). Nobody expected GMail to blow up the way it did, for example; it happened because some Googlers decided they could probably do a web-client-fronted mail client with a Google search engine attached to it and if they did it'd be really cool.

      But even with their, what, 180,000 people these days, I think it's entirely possible nobody is as excited about CRM as Paul Buchheit was about email services.

grumple 17 hours ago

I'm surprised, mostly because Google seems to have basically no salespeople, account reps, or customer management.

  • geodel 16 hours ago

    Google has ~40K people in sales and marketing. Likely they didn't assign any to your account.

    • wferrell 13 hours ago

      I have 3 for our google cloud account and we spend like 5k a month.

GHanku 19 hours ago

The linked article explains how they do it: https://www.bleepingcomputer.com/news/security/google-hacker...

>The attackers impersonate IT support personnel, requesting the target employee accept a connection to Salesforce Data Loader, a client application...

"The application supports OAuth and allows for direct "app" integration via the "connected apps" functionality in Salesforce," explains the researchers.

"Threat actors abuse this by persuading a victim over the phone to open the Salesforce connect setup page and enter a "connection code," thereby linking the actor-controlled Data Loader to the victim's environment.

... app is used to export data stored in Salesforce instances and then used the access to move laterally through connected platforms such as Okta, Microsoft 365, and Workplace.

Accessing these additional cloud platforms allows the threat actors to access more sensitive information stored on those platforms, including sensitive communications, authorization tokens, documents, and more.

1970-01-01 14 hours ago

Google: Nobody beats the $32,000,000,000 Wiz! Bet!

UNC6040: lool.