Fixing a Tiny Corner of the Supply Chain

No product gets built without at least one good supply chain war story – especially true in these strange times. Before we get into the details of the story, I feel it’s worth understanding a bit more about the part that caused me so much trouble: what it does, and why it’s so special.

The Part that Could Not Be Found
There’s a bit of magic in virtually every piece of modern electronics involved in the generation of internal timing signals. It’s called an oscillator, and typically it’s implemented with a precisely cut fleck of quartz crystal that “rings” when stimulated, vibrating millions of times per second. The accuracy of the crystal is measured in parts-per-million, such that over a month – about 2.5 million seconds – a run-of-the-mill crystal with 50ppm accuracy might drift by about two minutes. In mechanical terms it’s like producing 1kg (2.2 pound) bags of rice that have precisely no more and no less than one grain of rice compared to each other; or in CS terms it’s about 15 bits of precision (it’s funny how one metric sounds hard, while the other sounds trivial).

One of the many problems with quartz crystals is that they are big. Here’s a photo from wikipedia of the inside of a typical oscillator:


CC BY-SA 4.0 by Binarysequence via Wikipedia

The disk on the left is the crystal itself. Because the frequency of the crystal is directly related to its size, there’s a “physics limit” on how small these things can be. Their large size also imposes a limit on how much power it takes to drive them (it’s a lot). As a result, they tend to be large, and power-hungry – it’s not uncommon for a crystal oscillator to be specified to consume a couple milliamperes of current in normal operation (and yes, there are also chips with built-in oscillator circuits that can drive crystals, which reduces power; but they, too, have to burn the energy to charge and discharge some picofarads of capacitance millions of times per second due to the macroscopic nature of the crystal itself).

A company called SiTime has been quietly disrupting the crystal industry by building MEMS-based silicon resonators that can outperform quartz crystals in almost every way. The part I’m using is the SiT8021, and it’s tiny (1.5×0.8mm), surface-mountable (CSBGA), consumes about 100x less power than the quartz-based competition, and has a comparable frequency stability of 100ppm. Remarkably, despite being better in almost every way, it’s also cheaper – if you can get your hands on it. More on that later.

Whenever something like this comes along, I always like to ask “how come this didn’t happen sooner?”. You usually learn something interesting in exploring that question. In the case of pure-silicon oscillators, they have been around forever, but they are extremely sensitive to temperature and aging. Anyone who has designed analog circuits in silicon are familiar with the problem that basically every circuit element is a “temperature-to-X” converter, where X is the parameter you wish you could control. For example, a run of the mill “ring oscillator” with no special compensation would have an initial frequency accuracy of about 50% – going back to our analogies, it’d be like getting a bag of rice that nominally holds 1kg, but is filled to an actual weight of somewhere between 0.5kg and 1.5kg – and you would get swings of an additional 30% depending upon the ambient temperature. A silicon MEMS oscillator is a bit better than that, but its frequency output would still vary wildly with temperature; orders of magnitude more than the parts-per-million specified for a quartz crystal.

So, how do they turn something so innately terrible into something better-than-quartz? First I took a look at the devices under a microscope, and it’s immediately obvious that it’s actually two chips that have been bonded face-to-face with each other.


Edge-on view of an SiT8021 already mounted on a circuit board.

I deduced that a MEMS oscillator chip is nestled between the balls that bond the chip to the PCB. I did a quick trawl through the patents filed by SiTimes, and I’m guessing the MEMS oscillator chip contains at least two separate oscillators. These oscillators are intentionally different, so that their frequency drift with temperature also have different, but predictable, curves. They can use the relative difference of the frequencies to very precisely measure the absolute temperature of the pair of oscillators by comparing the instantaneous difference between the two frequencies. In other words, they took the exact problem that plagues silicon designs, and turned it into a feature: they built a very precise temperature sensor out of two silicon oscillators.

With the temperature of the oscillators known to exquisite precision, one can now compensate for the temperature effects. That’s what the larger of the two chips (the one directly attached to the solder balls) presumably does. It computes an inverse mapping of temperature vs. frequency, constantly adjusting a PLL driven by one of the two MEMs oscillators, to derive a precise, temperature-stable net frequency. The controller chip presumably also contains a set of eFuses that are burned in the factory (or by the distributor) to calibrate and set the initial frequency of the device. I didn’t do an acid decap of the controller chip, but it’s probably not unreasonable for it to be fabricated in 28nm silicon; at this geometry you could fit an entire RISC-V CPU in there with substantial microcode and effectively “wrap a computer” around the temperature drift problem that plagues silicon designs.

Significantly, the small size of the MEMS resonator compared to a quartz crystal, along with its extremely intimate bonding to the control electronics, means a fundamentally lower limit on the amount of energy required to sustain resonance, which probably goes a long way towards explaining why this circuit is able to reduce active power by so much.

The tiny size of the controller chip means that a typical 300mm wafer will yield about 50,000 chips; going by the “rule of thumb” that a processed wafer is roughly $3k, that puts the price of a raw, untested controller chip at about $0.06. The MEMs device is presumably a bit more expensive, and the bonding process itself can’t be cheap, but at a “street price” of about $0.64 each in 10k quantities, I imagine SiTime is still making good margin. All that being said, a million of these oscillators would fit on about 18 wafers, and the standard “bulk” wafer cassette in a fab holds 25 wafers (and a single fab will pump out about 25,000 – 50,000 wafers a month); so, this is a device that’s clearly ready for mobile-phone scale production.

Despite the production capacity, the unique characteristics of the SiT8021 make it a strong candidate to be designed into mobile phones of all types, so I would likely be competing with companies like Apple and Samsung for my tiny slice of the supply chain.

The Supply Chain War Story
It’s clearly a great part for a low-power mobile device like Precursor, which is why I designed it into the device. Unfortunately, there’s also no real substitute for it. Nobody else makes a MEMS oscillator of comparable quality, and as outlined above, this device is smaller and orders of magnitude lower power than an equivalent quartz crystal. It’s so power-efficient that in many chips it is less power to use this off-chip oscillator, than to use the built-in crystal oscillator to drive a passive crystal. For example, the STM32H7 HSE burns 450uA, whereas the SiT8021 runs at 160uA. To be fair, one also has to drive the pad input capacitance of the STM32, but even with that considered you’re probably around 250uA.

To put it in customer-facing terms, if I were forced to substitute commonly available quartz oscillators for this part, the instant-on standby time of a Precursor device would be cut from a bit over 50 hours down to about 40 hours (standby current would go from 11mA up to 13mA).

If this doesn’t make the part special enough, the fact that it’s an oscillator puts it in a special class with respect to electromagnetic compliance (EMC) regulations. These are the regulations that make sure that radios don’t interfere with each other, and like them or not, countries take them very seriously as trade barriers – by requiring expensive certifications, you’re able to eliminate the competition of small upstarts and cheap import equipment on “radio safety” grounds. Because the quality of radio signals depend directly upon the quality of the oscillator used to derive them, the regulations (quite reasonably) disallow substitutions of oscillators without re-certification. Thus, even if I wanted to take the hit on standby time and substitute the part, I’d have to go through the entire certification process again, at a cost of several thousand dollars and some weeks of additional delay.

Thus this part, along with the FPGA, is probably one of the two parts on the entire BOM that I really could not do without. Of course, I focused a lot on securing the FPGA, because of its high cost and known difficulty to source; but for want of a $0.68 crystal, a $565 product would not be shipped…

The supply chain saga starts when I ordered a couple thousand of these in January 2021, back when it had about a 30 week lead time, giving a delivery sometime in late August 2021. After waiting about 28 weeks, on August 12th, we got an email from our distributor informing us that they had to cancel our entire order for SiT8021s. That’s 28 weeks lost!

The nominal reason given was that the machine used to set the frequency of the chips was broken or otherwise unavailable, and due to supply chain problems it couldn’t be fixed anytime soon. Thus, we had to go to the factory to get the parts. But, in order to order direct from the factory, we had to order 18,000 pieces minimum – over 9x of what I needed. Recall that one wafer yields 58,000 chips, so this isn’t even half a wafer’s worth of oscillators. That being said, 18,000 chips would be about $12,000. This isn’t chump change for a project operating on a fixed budget. It’s expensive enough that I considered recertification of the product to use a different oscillator, if it weren’t for the degradation in standby time.

Panic ensues. We immediately trawl all the white-market distributor channels and buy out all the stock, regardless of the price. Instead of paying our quoted rate of $0.68, we’re paying as much as $1.05 each, but we’re still short about 300 oscillators.

I instruct the buyers to search the gray market channels, and they come back with offers at $5 or $6 for the $0.68 part, with no guarantee of fitness or function. In other words, I could pay 10x of the value of the part and get a box of bricks, and the broker could just disappear into the night with my money.

No deal. I had to do better.

By this time, every distributor was repeating the “18k Minimum Order Quantity (MOQ) with long lead time” offer, and my buyers in China waved the white flag and asked me to intervene. After trawling the Internet for a couple hours, I discover that Element14 right here in Singapore (where I live) claims to be able to deliver small quantities of the oscillator before the end of the year. It seems too good to be true.

I ask my buyers in China to place an order, and they balk; the China office repeats that there is simply no stock. This has happened before, due to trade restrictions and regional differences the inventory in one region may not be orderable in another, so I agree to order the balance of the oscillators with a personal credit card, and consign them directly to the factory. At this point, Element14 is claiming a delivery of 10-12 weeks on the Singapore website: that would just meet the deadline for the start of SMT production at the end of November.

I try to convince myself that I had found the solution to the problem, but something smelled rotten. A month later, I check back into the Element14 website to see the status of the order. The delivery had shifted back almost day-for-day to December. I start to suspect maybe they don’t even carry this part, and it’s just an automated listing to honeypot orders into their system. So, I get on the phone with an Element14 representative, and crazy enough, she can’t even find my order in her system even though I can see the order in my own Element14 account. She tells me this is not uncommon, and if she can’t see it in her system, then the web order will never be filled. I’m like, is there any way you can cancel the order then? She’s like “no, because I can’t see the order, I can’t cancel it.” But also because the representative can’t see the order, it also doesn’t exist, and it will never be filled. She recommends I place the order again.

I’m like…what the living fuck. Now I’m starting to sweat bullets; we’re within a few weeks of production start, and I’m considering ordering 18,000 oscillators and reselling the excess as singles via Crowd Supply in a Hail Mary to recover the costs. The frustrating part is, the cost of 300 parts is small – under $200 – but the lack of these parts blocks the shipment of roughly $170,000 worth of orders. So, I place a couple bets across the board. I go to Newark (Element14, but for the USA) and place an order for 500 units (they also claimed to be able to deliver), and I re-placed the order with Element14 Singapore, but this time I put a Raspberry Pi into the cart with the oscillators, as a “trial balloon” to test if the order was actually in their system. They were able to ship the part of the order with the Raspberry Pi to me almost immediately, so I knew they couldn’t claim to “lose the order” like before – but the SiT8021 parts went from having a definitive delivery date to a “contact us for more information” note – not very useful.

I also noticed that by this time (this is mid-October), Digikey is listing most of the SiT8021 parts for immediately delivery, with the exception of the 12MHz, 1.8V version that I need. Now I’m really starting to sweat – one of the hypothesis pushed back at my by the buyer in China was that there was no demand for this part, so it’s been canceled and that’s why I can’t find it anywhere. If the part’s been canceled, I’m really screwed.

I decide it’s time to reach out to SiTime directly. Through hook and crook, I get in touch with a sales rep, who confirms with me that the 12MHz, 1.8V version is a valid and orderable part number, and I should be able to go to Digikey to purchase it. I inform the sales rep that the Digikey website doesn’t list this part number, to which they reply “that’s strange, we’ll look into it”.

Not content to leave it there, I reach out to Digikey directly. I get connected to one of their technical sales representatives via an on-line chat, and after a bit of conversing, I confirmed that in fact the parts are shipped to Digikey as blanks, and they have a machine that can program the parts on-site. The technical sales rep also confirms the machine can program that exact configuration of the part, but because the part is not listed on the website I have to do a “custom part” quotation.

Aha! Now we are getting somewhere. I reach out to their custom-orders department and request a quotation. A lady responds to me fairly quickly and says she’ll get back to me. About a week passes, no response. I ping the department again, no response.

“Uh-oh.”

I finally do the “unthinkable” in the web age – I pick up the phone, and dial in hoping to reach a real human being to plead my case. I dial the extension for the custom department sales rep. It drops straight to voice mail. I call back again, this time punching the number to draw a lottery ticket for a new sales rep.

Luckily, I hit the jackpot! I got connected with a wonderful lady by the name of Mel who heard out my problem, and immediately took ownership for solving the problem. I could hear her typing queries into her terminal, and hemming and hawing over how strange it is for there to be no order code, but she can still pull up pricing. While I couldn’t look over her shoulder, I could piece together that the issue was a mis-configuration in their internal database. After about 5 minutes of tapping and poking, she informs me that she’ll send a message to their web department and correct the issue. Three days later, my part (along with 3 other missing SKUs) is orderable on Digikey, and a week later I have the 300 missing oscillators delivered directly to the factory – just in time for the start of SMT production.

I wrote Mel a hand-written thank-you card and mailed it to Digikey. I hope she received it, because people like here are a rare breed: she has the experience to quickly figure out how the system breaks, the judgment to send the right instructions to the right groups on how to fix it, and the authority to actually make it happen. And she’s actually still working a customer-facing job, not promoted into a corner office management position where she would never be exposed to a real-world problem like mine.

So, while Mel saved my production run, the comedy of errors still plays on at Element14 and Newark. The “unfindable order” is still lodged in my Element14 account, probably to stay there until the end of time. Newark’s “international” department sent me a note saying there’s been an export compliance issue with the part (since when did jellybean oscillators become subject to ITAR restrictions?!), so I responded to their department to give them more details – but got no response back. I’ve since tried to cancel the order, got no response, and now it just shows a status of “red exclamation mark” on hold and a ship date of Jan 2022. The other Singapore Element14 order that was combined with the Raspberry Pi still shows the ominous “please contact us for delivery” on the ship date, and despite trying to contact them, nobody has responded to inquiries. But hey, Digikey’s Mel has got my back, and production is up and (mostly) running on schedule.

This is just is one of many supply chain war stories I’ve had with this production run, but it is perhaps the one with the most unexpected outcome. I feared that perhaps the issue was intense competition for the parts making them unavailable, but the ground truth turned out to be much more mundane: a misconfigured website. Fortunately, this small corner of the supply chain is now fixed, and now anyone can buy the part that caused me so many sleepless nights.

34 Responses to “Fixing a Tiny Corner of the Supply Chain”

  1. Steven Ourada says:

    Great writeup. Thanks for telling the story; this is one of those behind-the-scenes peeks that people don’t usually disclose to an audience outside their company or immediate peers…

  2. Javier Cardona says:

    Very well written. Mel saved your day, but your persistence was also essential!

  3. mk says:

    OUTSTANDING!!! You still write an excellent thriller :-) I remember getting the first print of your book “hacking the Xbox” and reading it in a pub in Palo Alto. I spent about 3 days giving my chum at MSFT a hard time. Look what you did there, because of you we have kodi!!!

    I’m reading this in a coffee shop in downtown Chicago, surrounded by people hooked to their phones and tablets. I tend to carry a dumbphone so I get the chance to use my trusty ancient Ibm laptop running an operating system started by a kindred spirit of yours Linus.
    OUTSTANDING!!
    Mk

  4. JustSayin says:

    This sort of self-centered drama queen behavior is why I’ve always loathed “Bunnie the Great.”

    Dude. Catch a clue. You are a tiny, tiny fish out in the open ocean trying to compete with Apple, Samsung, Facebook, et al. Those guys are like, I dunna know, killer whales. They are the apex predators.

    When some new, cutting-edge component is available but in short supply, they can often—but of course not always—use their “convincing power” to cause suppliers to do their bidding.

    You can’t. You might have won this little battle but in general you are likely squandering your time tilting at windmills.

    I suppose Apple can and does get away with sole sourcing on occassion; but for you it’s a very risky endeavor. And, yeah, Mr. Engineering Super Genius, obviously you’ve decided to sole source a “bright shiny component” for this project. If you had a real manager over you, he would probably have nixed such a violation of Engineering 101 “Though shall not sole source… unless it is absolutely necessary.”

    As every first year engineering student should be taught in a proper university program, sole sourcing inevitably means that you are at the mercy of one supplier. That is an extremely precarious position to put oneself in.

    Unless you can continually convince armies of nerds to cough up huge premiums for products they don’t really need, then instead of geeking out over the latest “bright shiny tech component” which you can add to a new product, I suppose if probably would behoove you to use your remarkable engineering creativity to dream up useful products that could be made from more commonplace components.

    P.S. I used to try to argue congenially with engineers. However, because they are generally—and you particularly—are such a bunch of bigoted, arrogant, close-minded folks that I normally come out guns blazing from the get-go.

    • MWDC says:

      Ooh, an emotional hitman! Bunnie, you know you’re well when these kind of resources are being allocated to you ;)

      Keep up the awesome work everyone! Particularly Bunnie the Great.

      <3 MWDC

    • Jay says:

      “I used to try to argue congenially”, but now you are just a total jerk up front, saving everyone the time of figuring it out. Not arguing any of the actual points you made (obviously, arguing with someone like you is a waste of time, because you’ve made it clear you are “bigoted, arrogant, closed-minded folk”), but there is a difference between making consumer goods and something unique and cutting edge. So, yeah, your “Engineering 101”-level advice is spot on. Finally, “Dude. Catch a clue.” Sorry for “coming out guns blazing from the get-go”, but, ugh. You are not Linus, you can’t pull it off, lol.

    • mdel says:

      The good news is, taking this approach makes you look like the bigoted, arrogant, close-minded person. Congratulations!

    • Piotr says:

      Have you designed any product more complicated than blinking LED in the last 30 years? I have, a couple of them, in serious companies, for different markets, with quantities from 1000 to >100k. In any of them we managed to have multiple sources for all components. Not in a single one.
      You can design some projects that way, sure, but you won’t sell it. It will never be competitive, it will be bigger, draw higher power and/or will be more expensive than others on the market.

      • colin says:

        the point of multi-sourcing is to have redundancy when one supply falls through. i’m guessing you missed the part where Bunnie explains how he *could* drop in a crystal oscillator but would prefer not to unless absolutely necessary. you can do the exact same thing in classic multi-sourcing where maybe you trust the longevity of one flash manufacturer more than another. i think you’ve grossly misunderstood the set of tradeoffs at play here by eagerly labeling this as “single sourcing” without really thinking about what those words mean or why they matter.

    • Man I remember working with people like this. Fortunately they get written up by HR these days.

      • Tom says:

        Indeed JustSayin’s type is not accepted these days any more. Thankfully.
        Wonder how he’s doing in his fantasy world full of people he wants to surround himself with, exactly like himself, while collectively waiting for second-source versions of Xylinx FPGAs and Intel CPUs…

    • willmore says:

      Man, are you going to have a huge reality check when you get into actual design work. Good luck on your finals.

    • Akiba says:

      Just wanted to send a show of support. I’ve been a hardware and firmware engineer for more than two decades now and bunnie is one of the best engineers I know. The fact that he shares his experiences and mistakes is a huge benefit to the community and has helped me more times than I can count.
      Akiba

    • sjm4306 says:

      Reminds me of a saying: Better to keep your mouth shut than open it and be known as a bigoted, arrogant, close-minded a$$! You should really take your own advice and catch a clue that the reason you have so much social trouble with others is the common factor: YOU and not everyone else!

      Have fun alienating everyone, deservingly getting treated badly in return, and playing the victim in an attempt to rationalize all the drama you’ve created for yourself. Before you know it all you will have left is your hubris and ignorance.

  5. crd says:

    Maybe my brain just isn’t functioning yet but I’m not able to parse this:
    “there are also chips that integrated oscillator circuits that can drive crystals, which reduces power…”
    Can you clarify?
    Thanks…

    • bunnie says:

      That is my fault, I used the wrong tense for the verb. I have clarified it to “there are also chips with built-in oscillator circuits that can drive crystals, which reduces power”.

      In other words, instead of using an off-chip oscillator which incorporates its own active drive circuitry, there are chips that can directly drive the sliver of quartz. This saves a lot of power because the direct-drive oscillator circuitry can drive the crystal at a whisper of a voltage amplitude, while still achieving good noise performance.

      External oscillator circuits are often specified to create a CMOS-compatible output waveform, which is a full-swing square wave that also has to drive several picofarads of capacitance, an overhead which an on-chip oscillator circuit can forgo. That being said, not all on-chip oscillator circuits are born equal, and the ones that drive at a very low amplitude often have slow start-up times and are more sensitive to environmental factors.

  6. cgcm says:

    Great story that should help highlight an area that is generally not considered in the detail it requires… until your forced into that tight corner.

    Perseverance and Mel is all you need!

    Btw Jeff isn’t too bad in this area either, I’m sure you came across this but for others interested in “How to order custom SiTime Programmable Oscillators”:
    https://forum.digikey.com/t/how-to-order-custom-sitime-programmable-oscillators/5811/44

  7. Mrmowgli says:

    You know you have made it to the bigtime when you get passionate haters :)

    Great stories, and very helpful. Always glad to hear about your experiences, they open up a lot of these supply chain and manufacturing dark alleys. I’ve also experienced some of these zombied orders, and it’s nice to know how you got past yours.

    Big thumbs up!

  8. J. Peterson says:

    Fascinating story – I had no idea crystal oscillators are headed the way of film cameras.

    Out of curiosity I clicked the Digikey link for the part at the end of your story, and it shows “0 in stock”. But I think this means the parts don’t materialize until they program them.

    • willmore says:

      I can only imagine how much of a nightmare a part like this has to be for an inventory system as old as Digikey’s. They’ve learned to cope with blank parts that need custom programming. That’s something they’ve offered for years, but that was always a value added service which required back and forth with the customer, so it was always pushed to a different path. If you wanted to blank part, it was a standard listed part you could just order.

      Now you have a generic part which *use useless* unless its programmed by the distributer. There are ‘standard’ configurations for it, but they don’t keep programmed up parts on hand, but program them as needed. How is a poor inventory system supposed to deal with that? If it were up to me, I’d make all the derived parts show the same quantity as the ‘blank’ part, but I can see how that could cause all kinds of problems if you start generating reports on inventory value, etc. There would need to be a way of saying “show this # for quanitty when people are looking to order it, but show it as zero when counting inventory for cost of inventory reasons”. More and more special cases leading to more and more chance of mistakes.

      So glad I don’t work in that industry.

  9. Dave says:

    >The nominal reason given was that the machine used to set the frequency of the chips was broken or otherwise unavailable

    Was it made by Taylor Commercial Foodservice, better known as the manufacturer of McFlurry machines?

    • “otherwise unavailable” == the only person who knew how to operate it left, and they are willing to waste months trying to find someone to replace them who already knows the machine, so that they don’t have to be trained.

  10. グレェ「grey」 says:

    That was a fun, if harrowing read!

    In particular, the August 12th date jumped out. I think that was when I ventured out of my shelter-in-place bubble to see my first concert since the start of the pandemic?

    Since I made a road trip to Southern California for that show, earlier in the day attempted to track down an erroneous address for a different Crowd Supply order which had an off-by-1000 error in my zip code to no avail. I did at least find the street name collision in the urban environment, far, far, away, yet with no corresponding address collision and the neighbors with whom I inquired seemed really confused by my presence and questions. There have now been three attempted deliveries, a lot of questionable UPS data and claims and Crowd Supply email interactions. Alas, Crowd Supply have no phone number to call that I could find, so no hopes of reaching a Mel equivalent. Meanwhile I even went so far to attempt to communicate with their new parent company Mouser, which has also yielded little in the way of favorable results. Without shaming the project by naming it, that project purportedly shipped out in late May of this year and it’s now December and I still do not have what I preordered last year. Nonetheless, I am perhaps more concerned that whatever issues there were with that Crowd Supply order may trickle down into my Precursor preorders (somehow, some other Crowd Supply orders I placed since then, also inherited the address anomaly, despite the address error not occurring in the original preorder receipts). I have been assured that Crowd Supply have addressed the issues with my address? o.O Fingers crossed!

    The pandemic seems to have only exacerbated supply chain issues. I have a vague recollection that you may be a fellow deejay? Whether you are or aren’t, though not related to silicon, the Apollo Masters fire of 2020 has been rippling out in the vinyl production world in ways that were perhaps predictable to those paying attention to a dying physical format. It is now not uncommon to see announcements of 8+ month delays in album releases which I preordered. Thankfully, we live in an era where many recording artists master their compositions digitally and lossless FLACs can be purchased through some vendors. However, for die hard vinyl enthusiasts and luddite lathe engineers who never migrated to DMM mastering decades ago and instead decided to tough it out with lacquer back during the era when lathe manufacturers even had DMM machines to offer (which more or less ended in the 1980s near as I can surmise) as compact discs were coming into popularity? It is sort of like watching a collapse of a facet of an industry long past its prime play out in extremely slow motion, as if someone adjusted the pitch fader on their turntable to a very large offset. In other words, I guess it could be worse? I do not see any bright future ahead for vinyl, whatsoever.

    • A. says:

      You are not alone, I also had problems with Crowd Supply software: address parts mangled, etc.

      The funniest part was that the phone number on the label was taken from the billing address, then mangled so it became useless. Could not edit the billing address, so I had no chance to prevent the phone number corruption. The phone number that I fixed (to prevent mangling) in the shipping address was ignored.

      They were nice always and refunded or resent the products.

  11. Lev says:

    I was on the edge of my seat while reading this! Thanks Bunnie for sharing your experiences.
    The best engineers I’ve ever know were humble, and shared their mistakes/experiences openly.
    I’ve adopt this mentality myself (unfortunately, it wasn’t natural for me).

    Currently I am in the process of designing my first product (while starting a company) and reading
    your books/blog help a lot. Thanks again.

  12. Paul Farrell says:

    Bunnie – Sounds like a very familiar story to me and one I can repeat. In 2019 I was approached by a leading German company I supply mechanical parts to from my factories in Taiwan and China, to design, develop and manufacture in high volume a novel tire-pressure sensing device (TPMS). I can’t go into too much detail herein but was lucky enough to get a UK grant to help with proof-of-concept for my RFID+ FSR based device – It WORKS!
    Then the disasters started to happen. I tried, in vain, to locate a suitable UK manufacturing partner but got so bogged down in red tape I had to give up. The I tried some of the bigger component manufacturers here in Taiwan. Looked promising at first but then the “we won’t work with another company who isn’t one of our regular supply chain partners’ road block. That took another 6 months to try to figure out. next I found a promising RFID manufacturer who said they could take on the entire project – GREAT! We agreed a tentative ball-park mass production figure and price and off we went. 6 months later got a letter telling me the price had gone up x100 and they didn’t want the project any longer as they were “Too busy” with their mainstream RFID’s. Fuck me……
    So I’m back to trying to find somebody, anybody, who can take on this project with us, probably down in Shenzhen (where I used to live and work many years ago). But I’m kinda stuck now in Taichung with no local contacts. Really appreciate anyone following this blog who can reach out and hook me up. Cheers guys! And Merry Xmas

  13. […] Bunnie writes about “Fixing a small corner of the supply chain” […]

  14. Jered says:

    Yow! Good job, bunnie.

    It’s not relevant to this story, but my favorite thing about the SiTime oscillators is their sensitivity to helium. This blew up (heh) a few years ago: https://www.reddit.com/r/sysadmin/comments/9mk2o7/mri_disabled_every_ios_device_in_facility/

    Unsurprisingly, the MEMS oscillators are sensitive to the gaseous environment in which they operate…

  15. gpshead says:

    Gems like this are why we are all here!

  16. Alex Whittemore says:

    I’ve had similar problems with both RS and Newark just this past week. Nowhere near as enthralling or even consequential, but still a pain in the neck.

    The experience has actually given me a lot of assurance that distributors AREN’T actually fungible, and there really IS value to specifying parts DigiKey and Mouser carry, even if they’re more expensive there.

  17. 70sJukebox says:

    > I wrote Mel a hand-written thank-you card and mailed it to Digikey. I hope she received it, because people like here are a rare breed

    Absolutely, gestures like this do make a difference to the individuals involved.