Event Started: 9/7/2011

Hello and welcome to part two of our webinar series on HTA and Comparative Effectiveness Research (CER): HTA-CER-PCOR converging on what works for patients. Our speaker today is Cliff Goodman. Cliff is a senior vice president and principal at the Lewin Group, a policy consulting firm located in Falls Church, Virginia. Cliff has 30 years of experience working with government, industry, and nonprofit organizations in health care evaluation, primarily in health technology assessment and policy analysis with related expertise in evidence-based medicine, outcomes research, health economics, regulatory policy, third-party payment and technological innovation. Cliff is the Chair of the Medicare Evidence Development and Coverage Advisory Committee (MEDCAC), for the Centers for Medicare and Medicaid Services. He is President of the professional society, Health Technology Assessment international (HTAi) and is a fellow of the American Institute for Medical and Biological Engineering. He received a Doctor of Philosophy from the Wharton School of Pennsylvania, a Master of Science from the Georgia Institute of Technology and a Bachelor of Arts from Cornell University.

Thank you very much, Kate. I’m glad to be back for the second part of this webinar series. What we are going to do today is reintroduce and introduce some important concepts that in today's health care environment are converging in some pretty important ways. I will try to first of all decode the abbreviations on the title of this webinar. The main topics we are going to cover are these six; health technology assessment (HTA), comparative effectiveness research (CER), personalized medicine (PM), patient-centered care which is different from patient-centered outcome research (PCOR) and then some implications of the convergence of these important concepts. I am going to cover a lot of material today. We have nearly 50 slides but do know these will all be available after this webinar and I hope that you will return to them for more detail. I will say something about everything on all the slides that you are going to see. I will not go into them in great detail, but do hope you will return to them.

Let us move ahead. First of all, let us recap what we mean by health technology assessment. That is where we are going to start today. We devoted our previous webinar to this concept. Do remember that HTA involves a systematic evaluation of properties, effects, or other impacts of health care technology. Recall as well, this will arise a little bit later today that HTA has as its main purpose in forming health policymaking. It is not the same thing as the policy but it may inform or support policymaking. It addresses direct and intended consequences as well as indirect and unintended consequences of technology. It is conducted by interdisciplinary groups and uses a variety of analytical frameworks and methods. We will talk a little bit more about those today. When we talk about the sorts of things assessed by HTA, you will remember that there were several main categories. Today when we talk about personalized medicine, patient-centered outcomes research and so forth, we are going to focus quite a bit on breaking out efficacy and effectiveness a little bit more. Effectiveness in particular and how we assess effectiveness, especially from the standpoint of patient-centeredness and for the purposes of things like personalized medicine. Let us recall that important distinction between efficacy and effectiveness. Again, efficacy refers to how well something works under ideal conditions. This is often in the case of a carefully-managed randomized control trial. Typically, the results that we get from efficacy trials or ideal conditions are different and often better than what we get in effectiveness studies.

We are going to focus a lot today on effectiveness and that really regards the benefit of using a technology for a given health care problem in general. For routine conditions of use you will hear the phrase “real-world settings” or “realistic settings” and so forth. We are going to talk about effectiveness today, particularly from the standpoint of patients. When we look at how we measure efficacy or effectiveness, as you will recall, there are several main categories. At the top you will see health outcomes or end points. Sometimes we use the term “end points” when we look at benefits and harms. The main categories of health outcomes are end points, mortality, morbidity, and adverse events. Another set of dimensions involves quality of life, functional status, and patient satisfaction. More and more patients are getting involved in being assessed for these things; patient surveys and patient inputs. We are going to talk about how we look at health outcomes and end points for effectiveness in particular and from the standpoint of patients as it regards quality of life, functional status, and patient satisfaction. We will also look at intermediate end points such as blood pressure, lab values or EKGs. Sometimes we use the term biomarkers for things we can measure about a patient. Some subsets of these are called surrogates. Surrogate endpoints mean that they are highly and reliably predictive of health outcomes. Not all of these kinds of measures are true surrogates but if we have a true surrogate, we know that measuring that is pretty predictive of one of these outcomes above that we care about. Finally, if you are looking at things like screening diagnostic tests and monitoring tests when we talk about efficacy or effectiveness, we are typically talking about sensitivity and specificity (test accuracy). We are finding out there is more demand for understanding how a test can also be assessed for downstream effects on health outcomes as well. Do recall that there are three main groups or methods in HTA and these are going to translate or bridge to some of our concepts today, when we look at patient-oriented outcomes. Primary data collection involves things like randomized control trials, perspective or retrospective studies, observational studies, claims data, electronic medical records, various kinds of case series and any other way you decide to collect new data. Secondary integrative analyses are combinations; we synthesize or integrate data from existing sources. We do that in things like systematic reviews and analyses. Economic analysis was the third main group of studies we talked about before. We are going to talk today more about how we get at and what the interests are in data collection that reflects patient-centeredness.

Having talked a little bit about HTA, let us move to CER now. Why comparative effectiveness research? Here are a set of reasons why CER has emerged as being so important in recent years. Starting at the top is evidence of inappropriate use of health care technologies. We overuse stuff, we underuse Stuff, and sometimes we use it improperly even if we use it at the right time. There is quite a bit of evidence and literature supporting inappropriate use and describing it. Next, there is evidence of large variations in practice. A lot are familiar with the Dartmouth Atlas of the United States showing how the rate of use of various procedures, even for similar populations, is very different. Not everybody can be right about when to use technologies and those variations in practice raise red flags about the need for more evidence and more attention. Next, the evidence put together for FDA market approval in the United States in particular is usually more about efficacy than effectiveness and it usually involves narrowly-defined populations. That information about how well things work under ideal conditions or how a drug works versus a placebo under ideal conditions or in very narrowly-defined populations. That may be sufficient and appropriate for market approval by a regulatory agency, but that does not necessarily supply the information that we need in practice, that doctors need, that patients need, and even that payers and policymakers need. Therefore, that is an important distinction. Then, you have to realize that so many things that we do in health care are not regulated by the FDA and we do not even have good enough evidence that would even get them through the FDA. A lot of medical and surgery procedures are not subject to regulatory approval and do not have to do the kinds of clinical trials that most drugs and many devices have to undergo. I should say the evidence for that is quite inconsistent, insufficient and less rigorous in many instances. Next, there is lack of evidence on head-to-head comparison of alternative interventions of health care problems. Back to the FDA example; in most cases when new drugs are approved, they are approved based on studies comparing them to placebo. That is not always the case in areas such as HIV AIDS, cancer therapies, and so forth. We do have a realistic comparator but for many products, it is verses placebo. However, patient, doctors and payers say we are not using placebo, we want to know how your product compares to a standard of care. Therefore, we do not have enough evidence of what we call those “head-to-head comparisons”. Next, there is the lack of evidence in those real-world practice settings. Again, that is the effectiveness as opposed to efficacy and the difference is there is not enough effective data out there. Finally, given the rapid increases in health care costs, that is putting more pressure, attention, and scrutiny on ways that we can go about making the health care system more efficient.

There are six or seven reasons why comparative effectiveness research has emerged as being so important these days. If you wanted a good example of the lack of evidence on comparative effectiveness or even comparative efficacy for that matter, here is a good one. This was based on a systematic review of the strength of evidence for different kinds of radiation treatments of prostate cancer. You can see on the left-hand side, there is a whole range of kinds of radiation therapy interventions (all the abbreviations are explained below the table). These are the kinds of comparisons that you would like to know about if you were a patient or clinician or even a payer. When you ask what kind of evidence you have for those head-to-head comparisons; look what you find. In some cases in this first column, where we talk about freedom from biochemical failure, this happens to be a biomarker. This prostate-specific antigen is an indication for how the disease may be progressing and even for the biomarker, the evidence is almost always insufficient. For disease-specific survival, not a single one of these comparisons has evidence in the literature showing anything upon which we can make any conclusions. Also, the questions asked about adverse effects in this case, toxicity in the genital urinary tract or the gastrointestinal track, where quite a bit of adverse events can occur based on or due to the radiation therapy. To draw any conclusions about the evidence for the toxicity is almost always insufficient. In this case only for one comparison (comparisons among different kinds of external beam radiation therapy), there was moderate evidence for the biochemical failure and moderate evidence for the GU and GI toxicity. As you can see, the overall evidence on head-to-head comparisons was insufficient. This is a very, very good example. As a matter of fact, this systematic review, prepared by the Tufts’ Medical Center’s Evidence-Based Practice Center, was used by CMS to look at coverage issues having to do with these kinds of interventions. This is actually a presentation of evidence for policymaking.

What are the main attributes of comparative effectiveness research? First, direct or what we call head-to-head comparisons of alternative interventions, not just comparison with placebo or indirect comparisons. We are looking for head-to-head comparisons. CER is evolving to apply to all kinds of interventions. It is not just pharmaceuticals, biotechnologies, devices, or equipment; it is about medical and surgical procedures. If you look at priorities for comparative effectiveness research, they include things like organizational delivery management or even financing interventions. So CER can apply across the board to what we do in health care. It emphasizes effectiveness, health care outcomes such as morbidity and mortality, symptoms, quality of life, and adverse events, rather than emphasizing the immediate or surrogates. It does enable subgroup analysis. We need sometimes to break down data across larger populations into smaller sub populations to figure out how well something works for different types of patients, which is very important. Just as a footnote here, in the United States, CER typically does not involve economic issues. Some of that is social and political, yet other parts of the world where there are things like CER, tend to be explicit about involving economics. In any case, these are a half a dozen of the main attributes of what we recognize today as comparative effectiveness research. If you want to see definitions of CER, there are several good ones. I think probably the two most important are the ones I am going to the show you. Congress mandated the establishment of the Federal Coordinating Council for Comparative Effectiveness Research, which operated for about two years to help set the national agenda for CER. They have a two-part definition of CER. You will see some of the same words that involve comparisons, real-world settings and so forth. It really is designed to help improve outcomes and that is the first part of the definition. The second part gets into how you provide this information, what kinds of studies need to be conducted and to so forth. By the way, it does involve a variety of data sources and methods. That is a two-part definition from the Federal Coordinating Council. Another broadly recognized definition is that of the Institute of Medicine (IOM) and again you are going to see some of the same terminology here in these definitions; comparisons, real-world, subgroup and so forth. I think that there is a good sort of set or constellation of definitions of CER, but all those attributes of these definitions are captured here in this list of main attributes of CER.

Let us proceed past the definition then and just to give you little timeline here, I think an important thing to realize about CER, and a lot of folks that are working in the field have not had a chance to realize this year is that CER is not really a new thing. CER is a roll-up or convergence that borrows from types of inquiry that we have had around for a while. First off, you see the first randomized control trial was widely recognized and occurred in 1948. What does that bring? It brings an emphasis on experimentation, rigorous scientific approach, and trying to control for biases. Controlling for biases is an important quality. We show 1974 for health technology assessment because it happened in the year of the first report on HTA that came out of the Congressional Office of Technology Assessment. HTA says let us look broadly as we said before on the unintended consequences and the indirect consequences of technology. HTA said think broadly about what a technology can do and there are aspects of that, which roll up into CER. Outcomes research was widely recognized in the mid 1980s and in particular, by some work done by the agency that preceded the current Agency for Healthcare Research and Quality (first NCHSR and then HCPR). They really emphasized something on patient outcomes; not what we call biomarkers. How does something really affect patients? The science of assessing patient outcomes is really given a boost there. Effectiveness research was an initiative in the U.S. by the government. The predecessor agency to CMS said we want to know how well things work in the real world and so effectiveness was nicely recognized there. Pharmacoeconomics is for those of you who are interested in the economics side. This was one marker of the uptake or the diffusion of those kinds of studies. I think that 1989 was the first appearance of that term in the literature. Evidence-based medicine said, look, it is not just about the evidence alone; it is based on the evidence, but we also bring to bear the judgment and considerations of clinicians and patients talking to each other. So we are applying clinical judgment to the evidence in a particular situation for a given patient.

Comparative effectiveness research actually got formally started in 2003, in the Medicare Modernization Act, signed by President George Bush. They called for clinical comparative effectiveness and that provided some initial support to AHRQ and other groups to get rolling on the methods of CER and so forth. Of course the big boost for CER came in 2009, where $1.1 billion was set aside in the United States to really build up the national capacity for CER. In the mean time, coverage of evidence development (CED) came in 2006. Evidence development coverage was recognition that when we start to make decisions about using things in practice or even say something is regulated by the FDA; we do not always know everything we need to know. Payers, commercial payers, government payers and others have to make coverage decisions one way or another, but that does not mean that we stop looking for more evidence. In many instances, we made decisions to cover something on the condition that more evidence is generated because we do not always have what we need to know to make practice decisions and coverage decisions. We need to keep asking for evidence in many instances. That was CED or coverage of evidence development. In any case; when you consider what CER is today, it borrows those qualities that I mentioned from all these earlier instituted types of inquiry. By the way, all these types of inquiry are still in play and are very active. CER is a special kind of combination of some of the attributes of those other types of inquiry. When we do CER, we do not use any one single method; I like to say there is an evolving toolkit of CER methods. You will recognize a lot of these because we talked about them before in the context of HTA. A whole set of different trials and observations together comprise those primary data gathering studies. In other words, this is the process of collecting original data. You will remember that second category of syntheses, where they are used in comparative effectiveness research as well. We are still trying to understand how to adapt clinical trials and use observational studies to generate data on comparative effectiveness in real world health care settings and trying to make head-to-head comparisons. That is why I say that the methods toolkit is evolving. We still have to work on methods development and data development to use these. If you have a question about whether one type of intervention is better than another, you have to sometimes think of alternative data sources to try to get at data to help answer those questions. There is the CER methods toolkit that continues to evolve. By the way, some of the funding for CER, to which I eluded earlier, is going to help support development of these methods and tools.

A very important aspect of comparative effectiveness research that appears very clearly in the legislation and a lot of the literature surrounding the field has to do with providing attention of CER to these priority conditions. Very few of these, if any, should be a surprise you. They are what we call the usual suspects, involving high burdensome conditions that affect a lot of people for the most part. You do see they cover different kinds of populations and groups within the United States and around the world. CER is often focused and labeled as trying to address one or more of these priority conditions. This list was put together by the Agency for Healthcare Research and Quality (AHRQ) in response to requests by the Department of Health and Human Services (HHS) to help develop the Effective Healthcare Program there. These conditions remain priorities for CER. In addition to priority conditions, we have priority populations and those populations that are subject to health disparities. Here are commonly recognized priority populations, racial and ethnic minorities, people with disabilities, children, people with multiple chronic conditions and the elderly. We need more comparative effectiveness data and evidence that apply to these priority populations. Furthermore, many people are subject to unfair health disparities (significant gaps in the overall rate of disease incidence or prevalence), they are more at-risk for morbidity, mortality, and poor survival (includes some of these priority populations). When we look at CER, we are trying to think about how we can close these health disparity gaps. Priority populations, health disparities, priority conditions are all often the focus and the reasons why we need to do more CER.

One of the important aspects of furthering the national agenda for CER came from a mandate from Congress that the Department of Health and Human Services put together with its Federal Coordinating Council; it reports to the President and Congress. This came out in June of 2009. One of the most important aspects of this report to the President and Congress was a strategic framework for how we describe the field of CER and how we might invest in it. I think you will recognize some terms here. First of all, we need to deal with the CER research itself, that is, what kind of studies we are going to do in the field. But that is not it at all, that is not the only part of it. CER also involves investment in human capital (training and education) , CER and scientific capital, which refers to methods development and the development of analytical tools. Furthermore, we are strengthening the national data infrastructure for CER. For example, if we want to use large observational databases to do CER or put together registries or use electronic health records to conduct CER, we need to strengthen our national data infrastructure to enable that. Finally, if we get CER findings, we have to put them to use. There is a lot of work in dissemination and translation of CER to the various target audiences (users, stakeholders, and so forth). Notice that we care about the priority populations, priority conditions and types of interventions. Those cross all of the main “pillars” of CER. Therefore, this is somewhat of a high-level strategic framework describing the field. In particular, we think how about how we are investing in comparative effectiveness research; not just the research itself, but also the people, methods, data, and getting the message out there to make change occur. Another very important report, also mandated by Congress, is from the Institute of Medicine (IOM). Here is its National Priorities Report. The IOM’s report focused on generating one hundred national priorities for comparative effectiveness research. This is a very detailed exercise. The IOM received something like two thousand, six hundred comments and suggestions about what the priorities ought to be. What I think is very interesting about it has to do with starting from this list. These are the first priorities listed by the Institute of Medicine. These one hundred priorities were set out in the four tiers of twenty five; the first half dozen that are listed in the first quartile. Take a look at some of these; I think you will see some interesting diversity here. Immediately, you see the first one is comparison involving different kinds of interventions. Atrial fibrillation is the condition and so we are looking at comparing surgery, catheter ablation (a different kind of procedure), and treatment with drugs. We are looking at head-to-head comparisons of about three ways to attack a health care problem. The second one listed is quite interesting; it compares different treatments for hearing loss in kids and adults and looks at the kind of broader ways of closing that gap of hearing loss, in terms of the kinds of interventions involved. Another one that I think is also very interesting has to do with preventing falls in the elderly. If you are thinking about a health care problem that involves a lot of people with a great disease burden, comparing the effectiveness of primary methods for preventing falls is quite important. Looking at this, you will see they are comparing exercise and balance training, clinical treatments, and others. The IOM list of priorities, among many other things, illustrates the scope and extraordinary diversity of the field of CER and the kinds of things that comprise national priorities in the United States. Of course, many of these priorities are held in common in other parts of the world.

Now, we talked about HTA and we talked about CER. Also merging at this time, you will see where these start to interact, in personalized medicine (PM). Believe me; it is affecting how we think about HTA and how we think about CER. Personalized medicine involves the tailoring of medical care to the particular characteristics or circumstances of a patient that might influence that patient's response to some health care intervention. Now, quite often you hear about personalized medicine so for as genetics and genomics and of course that is an important part of personalized medicine. However, personal response to interventions may also entail socio-demographics, your clinical response type, behavioral aspects and characteristics, environmental, and even your personal preferences. Personalized medicine needs to account for what makes you an individual and how that individuality might influence how you respond to a particular health care intervention. It does not mean we are going to go about creating interventions, tailored for given patients that are unique to a patient. However, we do have to start thinking about how we might classify patients into subpopulations that differs their responses. When we collect data, we are thinking - are there some populations here that might respond differently and we might use that information about sub-population response to inform personalized choices. That is a key aspect of personalized medicine. Here are some commonly cited examples of personalized medicine and it is quite interesting that some of those are the genetic and genomic phenomena. These have to do with tests that find genetic polymorphisms or sometimes mutations that might affect how you might respond to a drug. For example, if you are on anticoagulation therapy because you have atrial fibrillation or a mechanical heart valve (where you are at risk for deep vein thrombosis), you might take warfarin therapy but certain people respond quite differently to warfarin therapy and that is often mediated by their genomic status. Tests for these polymorphisms might tell you and a clinician whether or not or how you might get warfarin therapy. Very widely known is the HER2/neu receptor testing for certain drugs for breast cancer. The BRCA1 and BRCA2 testing is not just for drug testing, but even surgical prevention options for breast cancer. KRAS testing is very important for those people with colon cancer that might be considering use of this group of drugs called EGFR inhibitors, such as the two shown there. KRAS testing is highly predictive of response to some of these drugs. It is not just these types; it is also things like individualized approaches or personalized approaches for something like bariatric surgery. There are different kinds of techniques for conducting this bariatric surgery and the one that is right for a given patient may depend on individualized factors. So that is an important example, I think, as well of personalized medicine. These are some of the more often-cited examples.

Going back to a similar timeline, you will see the light color shows the elements of the timeline that we showed for CER. We put those in the background, but how do we get to personalized medicine? There is a long history of developments here. Some key ones were the description of the structure of DNA by Watson and Crick back in 1953, the term pharmacogenetics that first appears in literature in 1959, and the genetic code was cracked in the late 1960s. Also, a key development was the so-called CYP450 metabolic enzymes that are mediated by your genome, effects how your liver metabolizes drugs, and therefore helps determine your response to those drugs. Of course the human genome was sequenced quite famously in 2003. I think sequencing in human genome really got a lot of folks excited about the potential for personalized medicine based on genomics and pharmacogenomics and so forth. The point of showing you this timeline is to say that the human genome was sequenced not far from the time CER got rolling. As a matter of fact, 2003 was the same year that the initial funding for clinical comparative effectiveness research was provided by the MMA, so these came together. Why is it so important that they co-occurred here? Well, first of all, let us talk about incorporating personalized medicine into the national CER priorities. I showed you a few moments ago the Institute of Medicine's list of national priorities for CER and in the top tier of those priorities is about personalized medicine. That priority looks at looking at comparing effectiveness of genetic and biomarker testing to usual care and how we prevent and treat different kinds of cancer. These types of personalized medical interventions are a part of national CER priorities. There are many others, by the way, in the list of the IOM that touched on things having to do with personalized medicine.

One of the important questions that arose as CER and PM appeared at about the same time was whether or not these are contradictory concepts. When you think about CER and other kinds of studies that evaluate technology, often times we think about what the average treatment effect is across a population. We are looking at population-based effects as opposed to personalized medicine which says, what is good for me? So some observers thought there might be some contradiction between CER and personalized medicine. Well, let us talk a little bit about why that could be a problem and then how we go about solving it. Like other forms of evaluation for health care interventions, CER generally has focused on identifying interventions that are effective across a broad population. The problem is that an intervention that yields a significant treatment effect across a study population on average may not work for all patients that might get this thing. As a matter of fact, it may be ineffective and harmful for others even though it had a statistically significant average treatment effect. On the other hand, you might say there may be an intervention that did not yield a treatment effect across a study population but it may work for some people. If it does not yield an average treatment effect that is seen as favorable, it might be dismissed even though some people or some population subsets might benefit from it. In short, that is the trouble with averages.

You may see this term”heterogeneity of treatment effects” and it is another way of saying what I just pointed out on the earlier slide. Start with the big curve; this big curve represents the population response across a population to some treatment. In this case, if you had no response at all (that is represented by zero), most of the population had a net treatment effect benefit. Some folks had a net treatment disbenefit. This is the universe of people that would respond favorably or not favorably to a treatment. If you look at this curve, there are a lot of people at this end of the curve that had an unfavorable experience and a lot people over here at this end of the curve did very well. If you make a decision based on just the average here, you might not be really helping these folks or reflecting their needs.

Across a broad population, you may have some mean treatment effect but that does not tell you enough about all these folks. Then, you think what if we are trying to detect different kinds of studies? How good are we at finding these so called heterogeneous treatment effects? In this sample, you see a subpopulation in a clinical trial. This sub population has the same mean treatment effect but as you can see, it is very narrow and it does not reflect people over here or people over here. So that may be providing only partial information. This subset does not reflect the diversity of the population. Now, another subset of the population that might fall over here, reflects those people that are at the upper end of the distribution and in the most favorable net treatment benefit effect. Again, that does not tell you much about the rest of the population distribution. We wonder when we do studies if we are finding these people ? Now this third sample, sample number three which is here, is more representative of the broader patient population. Its mean effect is the same as you can see, but it has a broader distribution or broader representation than sample one here. The main point here is to say that a broad population distribution is important to consider. You may have a net treatment effect or mean treatment effect but drawing all your conclusions based on this number is insufficient to represent these interests or these interests. Then, when you go back and do studies of subgroups, how good are you at representing the whole universe of population? That is a way of understanding heterogeneous treatment effects and why we need to think about patient subgroups to really understand what works in real life. By the way, here are some interesting earlier descriptions of the concept of heterogeneity treatment effects (HTE), although certainly Hippocrates was not using that term then but he certainly recognized the importance of how patients respond differently. Sir William Osler, a Canadian physician, also recognized that patients respond differently. I like what Hippocrates is said to have mentioned which was that “it is far more important to know what person the disease has than what disease the person has”. Again, he was not using the term HTE, but he was certainly thinking along the same lines.

How do we design comparative effectiveness research to reflect personalized medicine? In order to do that, we are going have to emphasize priorities and study designs that account for individuals' differences, as you can see here. These designs and studies need to be able to detect important treatment effects. We need to design CER in a way that accounts for these sorts of differences. You have to think about how big is my study, how representative it is of the population, and do I have enough people in these subgroups that I will have a big enough sample size to detect some of those treatment effects? When CER is designed, you need to keep in mind these aspects of personalized medicine. By the way, personalized medicine interventions are subject to the same rising evidence requirements that other technologies confront. That increasingly means showing that an intervention has a direct or at least a demonstratively indirect favorable impact in health outcomes in real world settings, not just biomarkers. You need it for genetic and genomic testing, which means not only demonstrating that the test is accurate, but that it has further downstream impact on health care decisions and health care outcomes. If you do not believe me, let us take a look at analytic validity, clinical validity, and clinical utility. It is not just for genetic tests, it is for a variety of tests. When you look at tests, the first question you might ask is how well does the test find the thing it is suppose to find (analytic validity)? For genetic testing, how good are you at finding that genotype? Genotype does not mean that you can also find the phenotype, which is clinical validity. Even if you can find the genotype, how well does that correlate with the expression of the associated disorder or clinical status or how well does it show up in the person (clinical validity)? Even if you can demonstrate analytic validity and clinical validity, that does not mean you have found clinical utility. Clinical utility answers how helpful the test is in affecting a decision and improving health care outcomes. If you stop here with the evidence, that is not enough. You have to ask about clinical validity and clinical utility. As a matter of fact, here is an evidence hierarchy. You may remember from the first HTA webinar that we talked about evidence hierarchies.

Here is an interesting evidence hierarchy that is used for genomics and you will see there are different levels of evidence for all three of those; analytic validity, clinical validity, and clinical utility. So we are pretty serious about the quality of evidence generated even for these kinds of tests. I am sure that molecular biology is cutting-edge science but when it comes to making decisions about patient care, coverage, and other issues, we still need evidence of high quality. Here is an example that takes that kind of framework and looks at it in an analytical approach in a clinical setting. In this instance, you might have a patient group here; these happen to be adults with non-psychotic depression who are starting therapies on selective serotonin reuptake inhibitors (SSRI). These are people that are candidates to take these kinds of drugs. We will run a test and this test may tell us if this person has this particular 450 genotype. That is helpful information because that kind of genotype is usually more likely to respond well to those drugs. That is not all the evidence we need. What evidence do we have that the genotype predicts your phenotype? How well you are going to metabolize those drugs? That is still not enough because we want to know how well it predicts the efficacy of the drug, let alone effectiveness, and how well it predicts an adverse event from the drug. In a way, this is kind of like analytic validity and clinical validity. However, clinical utility answers the questions, does this test affect a treatment decision, including information about harms and does this test tell us about improved outcomes? If you want to do well with the test, which might be a given here, we want to see downstream evidence of these effects all the way to does it affect a treatment decision and does it improve outcomes? That is what we mean by the downstream evidence. Those can be pretty high hurdles for tests but more and more clinicians and policymakers are asking for that kind of evidence. There are some details on the previous slide with regard to those various evidence questions. You can go back to that later.

Let us move now briefly to patient-centered care. We talked about HTA, CER, and personalized medicine. We are going to distinguish those topics from patient-centered outcomes. Patient-centered care is an important aspect of what we know about how the care focuses on the patients. This term appeared early on, perhaps in about 1970, and the contrast was between patient-centered medicine and illness-centered medicine. The contrast was also between an overall diagnosis that might be appreciated by a patient or patient centeredness, as opposed to a traditional diagnosis. Here are some very useful references of publications on this that I highly recommend to understand patient-centered care. Interestingly, we look at what the dimensions might be of measuring patient-centered care and you see four well-recognized dimensions for that, a disease and illness experience again for the patient, involvement of the whole person, common ground of patient and clinician, and how the patient and clinician relate. Here are some examples of some patient-centered care measures. There are others but these are some well-recognized ones; the patient perception of patient centeredness, instrument of consultation of care measure, and the Consumer Assessment of Healthcare Providers and Systems (CAHPS). CAHPS used to stand for the Consumer Assessment of Health Plan Survey. These are recognized ways of assessing the patient centeredness of the care itself. Now, one thing that is important to recognize about measuring patient-centered care is that while it is very important and we are talking about it a lot more, we still need some more work on measuring development. We recognize that there is no single measure out there that is going to suffice for everybody. We also recognize that patients, families, clinicians, and people in the health systems need to be involved in developing these measures. This last bullet point reflects an emphasis of CER, and the Patient-Centered Outcomes Research Institute, which is much more patient involvement in the tool development and assessments.

From patient-centered care, let us consider patient-centered outcomes. Essentially, patient-centered outcomes are outcomes that patients experience in real-world settings. That is the briefest, most to-the-point definition. These outcomes are patient-oriented, rather than disease or physician-oriented. Some of the main domains of patient-centered outcomes are these; health status, functional status, quality of life, quality of death, symptoms (including pain and nausea), and psychosocial wellbeing. These are the main sorts of aspects or names given to types of patient-centered outcomes. You may have also heard the term patient-reported outcomes (PRO). That term is often used in place of patient-centered outcomes. They do not necessarily mean the exact same thing but there is considerable overlap. If you are looking for some more examples in detail or the difference between a patient-oriented outcome and what might be a disease oriented outcome, you can refer to this list put together by Bell, et al., when they were looking at levels of evidence for measuring some of these things. For example, you might look at arthroscopic surgery for osteoarthritis of the knee, a disease oriented outcome might be something a clinician could appreciate and there might be improved appearance of the cartilage after debridement of the arthritic knee. That is fine and may be clinically relevant but the patient is not going to see what their cartilage looks like after debridement. What they are going to be concerned about is their functions or symptoms in, let us say, one year. There is a whole set of these things that are sort of side-by-side comparisons of more disease-oriented outcomes verses patient-oriented outcomes. The main point here is that although we still appreciate disease-oriented outcomes very much, we are increasingly looking for evidence of patient-oriented outcomes of the types listed here. Some examples of generic instruments for patient-centered outcomes in alphabetical order are; CAHPS, EuroQol, Health Utilities Index (HUI®), Nottingham Health Profile (NHP), Quality of Well-being scale, the Short Form-12® and Short Form-36®, and the Sickness Impact Profile (SIP). A lot of work is being done on these. I think Sickness Impact Profile goes back at least to the 1970s. Nottingham has been around for quite some time, as has the Health Utilities Index and the Short Form-12® series. There is a SF8® now, I believe. A lot of work is being done on all these and as far as validating them for different kinds of patients. Those are the generic instruments and it is interesting note that there are some condition-specific instruments for assessing patient-centered outcomes as well. Condition-specific instruments in areas such as angina, asthma, epilepsy, kidney disease, migraine, vision, and quite a few others. Typically, the assessment of patient-centered outcomes gathers data directly from patients, patient surveys, and other instruments designed for patients.

Let us talk about the formation of a Patient-Centered Outcomes Research Institute (PCORI). PCORI was formed to implement a national comparative effectiveness agenda, based on some of the earlier policies and legislation we talked about. It emphasizes patient centeredness, in particular with regard to CER. It was established by the Affordable Care Act in 2010 as an independent non-profit organization. Therefore, it was established by government but outside of government and it needs to identify research priorities, establish and implement a research agenda, and so forth. It has a 21-member board of governors and so forth. Note its mission is to help people make informed health care decisions. It is improving health care delivery and outcomes, evidence-based, and this comes from research that is guided by patients, as well as caregivers in the broader health care community. You can see again, thinking about our timeline, it is quite timely. If you have PCORI, what is patient-centered outcomes research (PCOR)? This is a good time to discuss this because just recently, PCORI put out a working definition of PCOR and it talks about helping people make those informed decisions. Notice the description here and the way they characterized it; it truly is patient-centered. Given one’s personal characteristics, what are their options? How do I improve the outcomes that are most important to me? How can health care systems improve my chances of achieving the outcomes that I prefer? Note the distinct patient-centeredness here of patient-centered outcomes research. Furthermore, PCORI’s definition proposed ways of answering these kinds of questions. Again, this is very revealing of the patient-centered aspect here. First, note they are looking at it across all kinds of interventions. This is quite consistent with how we define technology assessment of CER. Note comparisons, note outcomes that matter to people, note the inclusiveness of one's preferences, autonomy and needs. This needs to resonate with our understanding of personalized medicine, wide varieties of settings, diversity of participants, individual differences, and different types of burdens to people, resources and other stakeholder perspectives. This sort of explanation of PCOR really helps enrich our understanding of it in terms that we discussed before. It derives in part from HTA, CER, personalized medicine and patient centeredness.

Just a few weeks ago, one of the board members of PCORI proposed architecture of outcomes research that had some concepts that I am sure you are going to recognize, given our discussion. First of all, it incorporates the domains of comparative effectiveness, the patient perspective, and overall health system improvement. The kinds of inquiry are discovery science, which is really finding new data using methods and trials and observational studies. Discovery science is also how things are applied in care and studying what happens in real practice and ongoing surveillance. The research itself asks what is being achieved for patients and how can we do it better? You will recognize this focus on outcomes that patients experience in the real world. All of this heads toward what Information improves clinical decisions and policies and it has to apply to health care practice to improve patient outcomes. I think this is a very nice framework or architecture of outcomes research and I hope you will appreciate that it captures these domains and principles that we have talked about thus far. Here again is our friendly timeline. Superimposed on what we learned earlier about CER and personalized medicine, you see these benchmarks in patient-centered medicine. We bring up the one in outcomes research again, which I showed you originally with CER because the work and outcomes research had a great deal to provide to the basis and methodological underpinnings of patient-centered outcomes research. You will notice that PCORI was established in 2010 and even now we are working on a definition for patient-centered outcomes research in 2011. I hope you will appreciate how all of these concepts have come together and converged in a very interesting environment today for understanding what works today in health care, especially so far as it focuses on what is good for patients.

Let us do a little summary of the implications of HTA, CER, personalized medicine and patient-centered outcomes research. I will not go over these in great detail but I will give you the highlights. First of all, evidence standards are generally rising and being used more broadly. We care about evidence and outcomes, especially patient-oriented ones. We care about real world settings, we care about patient subgroups and heterogeneity treatment effects, and we care about priority populations and those subject to health disparities. We need evidence across all these. There is increasing acceptance of non-RCT observational data for certain kinds of evidence questions but we need more work to develop those. We want to develop better and more predictive instruments to measure these aspects; measure patient-centered care, measure patient-centered outcomes. Yes there is a lot of emphasis on patient centeredness but we need the methods and tools to support that. What I think is also very important is this stuff is applying not just to drugs, devices and surgical procedures; it is applying to organizational delivery management and financing interventions. All of these things contribute to patient experience. PCORI in the United States is focusing a great deal of attention on patient-centeredness, including patient-stakeholder involvement and investment in data resources and methods to support PCOR. We are redefining value and shifting the direction of innovation. This is an important factor arising from this convergence. These trends and demands for data are really affecting choices about developing technologies and they are influenced by a variety of factors. We need to now recognize that those who have developed new technologies and so forth (whether it be pharmaceuticals, devices, biotechnology, surgical procedures, electronic health record systems), we need to validate technology. We need more head-to-head comparison with real health outcomes data in real settings. Given the more cost-constrained health care system, that is changing innovation a little bit in how we consider what comprises value in health care. In the pharma-bio industry, in particular, where that Block Buster model of certain pharmaceuticals is not working so well for various reasons, there is greater potential in high value treatments for certain patient subgroups that we identified through genomics around personalized medicine. That is to the extent we can develop interventions of great value. Even if they are for more narrowly defined patient subgroups, we can achieve high value for those. That factor is something that is affecting decisions about innovations in health care.

We will start wrapping it up there. I hope you will recognize that it is a really exciting time today in healthcare. Over the years in the fields of HTA, CER, personalized medicines, and patient-centered outcomes research - they are converging where we can capitalize on what we are learning, we can develop new evidence, we can develop new ways of understanding how patients interact with the health care system, and how they benefit from it. We try to focus these efforts in ways that will generate interventions that help patients, families, caregivers and others achieve improved health care outcomes in today's world of health care as well as the future. I will stop there; I appreciate your time today on this part two of our webinar series HTA.

Last Reviewed: February 23, 2024