Why the Fleischner Society Criteria Kind of Suck!


I don’t mean to be too inflammatory here but let’s face it, when a set of guidelines that was agreed upon across specialties, that’s fairly easy to memorize (only one variable), and that was published some 10 years ago STILL doesn’t have it’s recommendations routinely reported in the majority of radiology reports, well, we’ve got a problem.

And I don’t place the blame at the feet of the Fleischner Society.  Rather it’s the guidelines themselves…all guidelines…and the Fleischner Society guidelines are just the most obvious example.  Guidelines exist to be adopted and affect care on a large scale.  They create a sort of “magnetic line” to practice around.  Unfortunately most published guidelines just don’t accomplish that goal in a timely or effective manner…so that is what sucks!

At the recent RSNA, we presented our experience launching RadsBest, and some of what we’ve learned from our many users.  I think the most important thing that we learned is that many of these guidelines…well, they are just not being adopted in the way that they could be, and there are important reasons why they aren’t.

Here is a portion of the presentation that we made.  It contains our breakdown of the factors that have affected the adoption of clinical management guidelines like the Fleischner Society Criteria and ACR Management Guidelines for Incidental Findings.

Read on.

When publishing guidelines for clinical management, there is a constant tension between creating something that is useful on a practical level (i.e. an algorithm that is not too onerous to memorize or navigate) and something that is applicable to a wide variety of patient populations and that provides enough complexity to optimize care for each individual patient.  Unfortunately when you publish such a guideline, you often have to make tradeoffs between the two.  Decision support technology, like RadsBest, is able to eliminate that tension between the usable versus the accurate by putting the complexity in the backend.  It allows creation of algorithms that are BOTH Usable AND Accurate.


As of November RadsBest has had downloads from around the world.  There were just under 2000 registered users…even more now.


Interestingly, the two of the three most popular tools were both published WAY back in 2005!  The top tool is the Fleischner Society Criteria for solitary pulmonary nodules, which supports the fact that many radiologists are probably not fully memorizing even the simplest of guidelines (including myself).

Importantly, there are quite a few radiologists who download the app but do not actually use it much beyond an initial perusal.  This drop off in user activity is a common phenomenon with any product but still provides insight.  Why there is such a dropoff in user activity?

Most obvious explanation for the gap between registered users and “active” users is that the app, or what it does, is just not that useful or desirable…or the app doesn’t work well.  This is why we are extremely interested in feedback from our users (both frequent users and users who never use it). We are always working on improvements that will make the app an indispensable tool.

But if you think about it a bit more, there is another factor to explain that drop off in activity.

RadsBest is really the only available app that does what it does.  Management decision support for the practicing radiologist.  Therefore, it is reasonable to assume that the drop off in usage is also a coarse surrogate for how much radiologists are actually adopting these management guidelines in their daily practice by putting those recommendations into their reports.

Here is a simplified breakdown of the factors that affect the adoption of a management decision support tool like RadsBest.

This is probably the most important slide of this talk.  The “Buy In” I believe is the biggest issue facing radiology when it comes to implementation of a more standardized method of findings management.  You need “buy-in” from your local radiologist, who has to make a conscious decision to utilize management guidelines in each and every report.  But, more importantly, you need “buy-in” from your clinicians as well…and this is not just a philosophical “buy-in”, there are often financial externalities that complicate matters.  Where I practice, the referring physicians are under intense pressure from their IPAs to cut costs, including imaging.  They may not agree that pulmonary nodules should be followed at the same frequency as that dictated by the Fleischner Society because that will result in a lot of CT scans that need to be paid for.

Also, a lot of the guidelines that are currently being advocated by the American College of Radiology (ACR) are just not known to our referrers.  It is a double-edged sword but many of these guidelines were not created in conjunction with other stakeholder medical specialties.  This is an obvious factor in getting national level “buy-in”.  That is the level of support that would be needed in order to get widespread adoption of guidelines such as these.

An app like RadsBest (or anything that provides a similar function) can actually help to drive the culture shift within medicine.  It can do so by not only making it easier to use the guidelines, thereby diminishing the barrier to entry, but also by making the guidelines more visible.  There are many guidelines collected together within the app and I would bet that most radiologists were not aware that they all existed.

We are working hard to make a better product.  We have great ideas for improvements for the future and really welcome feedback from our users on how we can make the app even better and even more useful.  Please drop us a line through the feedback link in the app or leave a comment below.


RadsBest in the Spotlight at AuntMinnie


Just did an interview with AuntMinnie.com for their Mobile App Spotlight.  There are some major typos (come on editors!), but it provides a little insight into why we created Medocratic and what we are doing with RadsBest.  Here are a few key quotes:

At the pace that radiology happens nowadays, it’s a struggle to just provide useful recommendations, let alone recommendations that are optimal and also consistent.

Things are moving so fast and systems are so complex that when it comes to introducing new technologies — the same ones that you depend on day in and day out — the actual practicing doctors who will be using them often don’t have enough input, or the systems are so complex that even with said input you can’t really know what you are getting until it is too late to turn back.

Check out the rest of the interview HERE.


RSNA 2014 Here We Come!

As part of the scientific program at RSNA 2014, our Founder, Roy Kwak, M.D. will be presenting:  Adoption of a Mobile Clinical Decision Support Tool for Radiologists

In his talk he will detail the experience of launching RadsBest and the data/conclusions/lessons learned from the process.  Mark it on your calendars, we’d love to meet you!

Mon Dec 01 2014 11:20AM – 11:30AM ROOM S404CD


Why Did You Start Charging Money for Tools?

Some people may wonder why we made the decision to charge for some of our tools.

When it came down to it, it was an easy decision.

We are physician owned and physician run. It is our primary mission to deliver tools free of B.S.– No ads, no pharma, no corporate sponsors, no directives.

Developing software isn’t cheap. Developing software of the quality that physicians expect is downright expensive.

That’s it. If we want this platform of tools to continue to grow and be relevant to daily practice, we need to be able to keep the lights on.

Keep reading if you want to know more about our decision making process and how the in-app purchase system works.

RadsBest has always been a passion project.  It was a simple problem in need of a solution.  The very first version of RadsBest was developed for the web.  Hiring web developers was relatively cheap in 2012, with native iOS developers charging 3-4 times what web developers were charging.  When we worked out the backend kinks on the more forgiving html environment, we decided to jump into the frigid waters of mobile development.  We partnered with an entirely new development team and created Version 1.0 of RadsBest.  Remarkably, we were able to do this at a fraction of the cost of many similarly scaled apps.  Despite our luck in this regard, it was very clear that we were spending real money.  It was also very clear that we would never be able to create many of the cool features that we always envisioned for RadsBest unless we could somehow make the app more sustainable and prove that the market existed for tools such as this (i.e. tools with a low barrier to entry that help self-motivated docs do their jobs “better” and “faster”).

The new version of RadsBest incorporates in-app purchases.  This allows the user to purchase any number of Credits within the app using real money.  The Credits can be redeemed for new tools within the app itself.  The beauty of the Credit system is that it gives us, as developers, a ton of freedom with pricing.  We don’t have to sell everything in 99 cent increments.  We can charge 1 cent to $100 for an individual tool.  It also allows us to provide some positive feedback to our users.  For example, all users who participated in the Beta Testing when RadsBest was first being developed will get more free credits than they will ever be able to use within the app (we keep our promises).  This will allow them to download any tool that we develop without paying a penny in real money.  We are also pretty generous when it comes to good user feedback (i.e. bugs within the app or thoughtful suggestions for new tools).

In the end, the ability to purchase tools within the app will help development of RadsBest and allow us to reach more Radiologists with increasingly useful and functional tools.

Roy Kwak, M.D., Founder, Medocratic LLC


Rad’s Best Friend is now Beta Testing!

I’ve spent a lot of time in the past few months developing a web based application that is meant to help Radiologists raise the bar for quality. This program is designed to improve the accuracy and consistency of your reporting, including providing standards-based follow-up recommendations and TNM staging for many commonly encountered clinical scenarios. I’ve based this on widely accepted consensus papers published in our major journals and in ACR white papers. Although many of these guidelines are known by radiologists, study after study has shown that most radiologists are not utilizing them…not even the Fleischner Society Criteria for Pulmonary Nodules, which is the most simple guideline out there.

So enough talk. The program is called “Radiologist’s Best Friend” and it is currently in beta testing. It will have many more modules when it is eventually released as a full-fledged app. You are invited to come and try out the web-based version. The plan is to release this as an iPhone application (which will drastically improve speed and obviate need for mobile internet access).

In return for your feedback I am offering the program “FREE FOR LIFE”. That means that when the program is released for the iPhone (and possibly Android) later this year you will get it for FREE, along with any upgrades (future versions, etc.) for the life of the product. Here is what you need to do to get this:

1) Go to the link: http://beta.radbf.com/

2) Click the “OK” button on the first page, which will take you to a menu page. There is a button at the bottom of this page titled “FIRST SURVEY”. Press that button and take the survey (it will popup as a new window). Make sure you fill out your name and email because that is how I will track your “credit” for the future.

3) Then, just spend some time using the program as you read your cases. Try and see how it would fit into your normal workflow. The program itself should be pretty self explanatory.

4) After a couple days, click on the Survey Button that pops up on the bottom of the “recommendation” pages. Make sure you fill out your name and email because that is how I will track your “credit” for the future.

5) Hit the facebook “Like” button on the menu page.

5) That’s it. Two surveys and you will get this program for free from now until “forever”.

Feel free to share this link with anyone else you may know who may appreciate a program such as this. This offer extends to them as well.