[00:00:00] Speaker 02:
Case number 20-5035, Pavement Coatings Technology Council, a balance versus United States Geological Survey.

[00:00:08] Speaker 02:
Mr. Ebner for the balance, Mr. Koppel for the appellate.

[00:00:13] Speaker 02:
Good afternoon.

[00:00:14] Speaker 04:
May it please the court.

[00:00:16] Speaker 04:
I'm Lawrence Ebner and I represent the Pavement Coatings Technology Council.

[00:00:21] Speaker 04:
USGS is asking you to expand FOIA Exemption 5 in a way that would insulate all federal government scientists from external criticism of their published research.

[00:00:33] Speaker 04:
According to USGS, a government scientist can hide behind the deliberative process privilege simply by labeling

[00:00:43] Speaker 04:
essential study-related data as exploratory analysis.

[00:00:47] Speaker 04:
If USGS is correct that the deliberative process privilege applies to scientists thinking virtually all federal government research is going to be covered by an exemption that is supposed to be construed narrowly.

[00:01:04] Speaker 04:
We believe that concealing scientific data so that a study cannot be discredited is not the purpose of the deliberative process privilege, for example.

[00:01:15] Speaker 02:
That doesn't make clear.

[00:01:16] Speaker 02:
You have the data itself, correct?

[00:01:19] Speaker 04:
The only data that we have are the raw laboratory data.

[00:01:23] Speaker 02:
That's exactly what I'm referring to.

[00:01:25] Speaker 02:
You have the raw data.

[00:01:28] Speaker 02:
What you do not have is two people

[00:01:34] Speaker 02:
who have analyzed this data.

[00:01:40] Speaker 04:
What we do not have and what we.

[00:01:42] Speaker 02:
The four out of 200 options they have here.

[00:01:52] Speaker 04:
That's correct.

[00:01:53] Speaker 04:
We do have, and government counsel will tell you, USGS has produced tens of thousands of pages of raw laboratory data.

[00:02:02] Speaker 04:
Those data are primarily late sediment chemical analyses and measurements.

[00:02:08] Speaker 04:
They are useless to us without the computer modeling inputs and outputs that we seek.

[00:02:16] Speaker 04:
In this case,

[00:02:17] Speaker 04:
Many state and local governments have relied on the urban lake study publication to ban the use of refined tar sealant.

[00:02:26] Speaker 04:
The published study does disclose the computer modeling input and output data only from what study authors describe as their four best modeling scenarios.

[00:02:37] Speaker 04:
Those are computer modeling.

[00:02:39] Speaker 03:
All of the raw data.

[00:02:41] Speaker 03:
And you have these four models.

[00:02:46] Speaker 03:
Why on earth couldn't your own scientists with all the rod.

[00:02:50] Speaker 03:
They know the question.

[00:02:51] Speaker 03:
They have all the raw data.

[00:02:53] Speaker 03:
They know what models were used.

[00:02:55] Speaker 03:
They could run it and then they could tweak and redesign the models to test Just like these scientists did.

[00:03:03] Speaker 03:
That's what scientists do.

[00:03:04] Speaker 03:
Why can't you do your own model runs with the same data.

[00:03:09] Speaker 04:
Judge Millett, I think that one of the study authors, Dr. Van Meter, answers your question in his declaration where he explains that there were so many possible combinations of inputs for a given model that the possible outcomes are virtually limitless.

[00:03:29] Speaker 03:
And in fact, that would change if they give you the other model runs.

[00:03:34] Speaker 03:
So I'm trying to understand why, if your concern seems to be

[00:03:38] Speaker 03:
What your, what your FOIA interest here is knowing what other models were tried, what changes, what tweaks or adjustments were made to these four models that they didn't use, they didn't treat as their four best models.

[00:03:56] Speaker 03:
And any scientist can come up with their own tweaking to models, try their own changes.

[00:04:02] Speaker 03:
They can take all those variables and make all those changes.

[00:04:05] Speaker 03:
So you don't need

[00:04:07] Speaker 03:
You may want it, but you don't have a need for that information to understand what the government did.

[00:04:13] Speaker 03:
You can test the government's theory because you have the data.

[00:04:17] Speaker 04:
With respect, your honor, all we can do with the data that has been provided is to try to replicate what the study authors describe as their four best modeling scenarios.

[00:04:28] Speaker 04:
Modeling runs.

[00:04:29] Speaker 03:
Your scientists aren't capable of designing their own model runs?

[00:04:33] Speaker 04:
There are so many possible combinations that what we want to do, Your Honor, is take a look at the other 200 modeling runs that the study authors chose not to use in their published study and not to disclose.

[00:04:52] Speaker 04:
They're saying in their published study, trust us, those other 200 modeling runs

[00:04:58] Speaker 04:
are going to confirm the four modeling runs that are discussed in the published study.

[00:05:05] Speaker 04:
In order for us to replicate the other 200 modeling runs, we need not just the raw data, but we need to have the input and output data so that we can run those other 200 models and see, for example, whether they report the... You have the data that they put in and you have the data that came out.

[00:05:27] Speaker 04:
We do not, Your Honor.

[00:05:29] Speaker 03:
I thought you had all the raw data.

[00:05:31] Speaker 03:
Are you saying there's raw data that wasn't disclosed that they input into their model runs?

[00:05:35] Speaker 04:
There's a distinction, Your Honor, between the raw data and the computer modeling input and output data that we seek.

[00:05:43] Speaker 04:
USGS on page 12 of its brief explains that modeling data are critical, that's their word, critical to scientists' ability to analyze the raw data.

[00:05:54] Speaker 02:
You have the raw data, and you have their four runs, right?

[00:06:00] Speaker 04:
What we want to do is duplicate.

[00:06:03] Speaker 02:
You want to do the hundred and what?

[00:06:06] Speaker 02:
186, whatever.

[00:06:07] Speaker 04:
Approximately 200 modeling runs that they ran, but

[00:06:11] Speaker 04:
but won't tell us what those modeling runs are.

[00:06:14] Speaker 04:
And in order for us to test the study's conclusions based on their four best modeling scenarios, we need to know what all the parameters were, not just the raw data, but the parameters

[00:06:30] Speaker 04:
and the combinations of parameters, which Dr. Brand-Meter says are virtually limitless, in order to see how those other two modeling runs come out and see whether they really do support what's in the published study or whether they undermine it.

[00:06:44] Speaker 01:
And I'd like to point out- In other words, you want to be able to see whether in their deliberations, they deliberated and reached the best conclusion or not, right?

[00:06:59] Speaker 04:
Well, I wouldn't call that deliberations, but I would like to know whether their study, for example, suffers from confirmation bias.

[00:07:13] Speaker 01:
Yeah, I mean, my point, sir, is that to say that you aren't really seeking to understand the deliberative process of these scientists seems to be

[00:07:28] Speaker 01:
like saying that there's no gambling going on in Casablanca.

[00:07:33] Speaker 01:
I mean, the whole purpose of this request is that you want to undermine their deliberative process because you think that it has been infected by bias or incompetence or whatever, right?

[00:07:51] Speaker 04:
Well, yes, Judge Wilkins, but the scientific deliberative process, which is really the scientific method, is not the same as the deliberative process that's contemplated by FOIA Exemption 5.

[00:08:03] Speaker 01:
Okay, so then let's get down the brass tacks.

[00:08:07] Speaker 01:
You do want to invade their deliberative process, but you believe that scientific deliberative process is not covered because that's not policy.

[00:08:19] Speaker 01:
deliberative process, right?

[00:08:22] Speaker 04:
It's not only not policy, but it falls within the distinction that the Supreme Court and the DC Circuit has drawn for many years between purely factual

[00:08:36] Speaker 04:
information and drafts and so forth that go into the formation of policies and recommendations.

[00:08:45] Speaker 04:
We are not seeking, for example, any notes or memos or emails that these scientists may have generated that analyze or evaluate or discuss the modeling runs.

[00:08:57] Speaker 04:
We're not asking for drafts of the proposed study.

[00:08:59] Speaker 04:
We're not asking for internal USGS comments on the proposed study.

[00:09:04] Speaker 04:
What we are asking for

[00:09:05] Speaker 04:
is pure scientific data, which are contained on spreadsheets, charts, and graphs.

[00:09:10] Speaker 04:
And I might point out that USGS's own internal manual defines data as non-interpretive.

[00:09:18] Speaker 04:
And I'd like to refer the court to the American Radio Relay case, because in that case, the court held that a federal agency cannot invoke

[00:09:31] Speaker 04:
FOIA Exemption 5 in order to hide parts of a study that are inextricably bound to the study as a whole that may contain contrary evidence or may help to eliminate the strengths and weaknesses of the study.

[00:09:46] Speaker 02:
Disclose the whole study.

[00:09:49] Speaker 02:
So you have all the data.

[00:09:51] Speaker 02:
You have the final study with the four runs.

[00:09:56] Speaker 02:
And you want the other 196, which I gather from your statement, you're saying the agency still has.

[00:10:08] Speaker 04:
They still have.

[00:10:09] Speaker 04:
They refuse to produce them.

[00:10:11] Speaker 04:
There's nothing inherently deliberative about them.

[00:10:14] Speaker 04:
If these kinds of data, input and output data, were inherently deliberative, they wouldn't have disclosed the inputs and outputs for what they call their four best modeling runs.

[00:10:24] Speaker 04:
And to use the word that USGS uses in its brief, what we see, the input and output data for the other 200 modeling runs are critical to the ability to analyze the raw data that they produced.

[00:10:38] Speaker 04:
They basically dumped tens of thousands of pages of raw data on us and said, we should go figure out by an infinite number of permutations what their other 200 modeling runs were, because they won't tell us.

[00:10:53] Speaker 04:
And we don't think that's what's contemplated by Fourier Exemption 5.

[00:10:57] Speaker 03:
To be clear about something, you already have the data.

[00:11:00] Speaker 03:
It's just not labeled input and output.

[00:11:03] Speaker 03:
Or are you saying there's raw data you don't yet have?

[00:11:07] Speaker 04:
Well, I'm using the term raw data like the study authors do, which is the lake sediment measurement, which is one of the inputs into the computer model.

[00:11:18] Speaker 03:
You have all the data.

[00:11:20] Speaker 03:
It just doesn't come with a label input and output.

[00:11:22] Speaker 04:
No, your honor, we do not have the input and output data.

[00:11:28] Speaker 04:
Another term for that is the modeling runs.

[00:11:31] Speaker 03:
Okay, that's different.

[00:11:32] Speaker 03:
Then let's let's just call it model runs versus data.

[00:11:34] Speaker 03:
You've got the raw data.

[00:11:35] Speaker 03:
Let's not call model runs data because I think that's

[00:11:38] Speaker 03:
question in its own right.

[00:11:40] Speaker 03:
So let's just call it model runs instead of saying you don't have certain data.

[00:11:43] Speaker 04:
Model runs represent the inputs and the outputs.

[00:11:47] Speaker 04:
There were more than 200 of them and USGS has refused to produce them and we cannot test the urban lake studies conclusions without those other 200 model runs.

[00:11:58] Speaker 04:
We cannot do it just with tens of thousands of pages

[00:12:01] Speaker 04:
of raw laboratory data.

[00:12:03] Speaker 04:
We cannot do anything.

[00:12:05] Speaker 03:
The articles were peer reviewed before publication in the journal.

[00:12:09] Speaker 03:
Does the peer review process require disclosure of all 200 model runs?

[00:12:16] Speaker 04:
We don't know whether those 200 model runs were disclosed, but in our brief, we cite, for example, nature publications policy, which does require scientists to make all the data available.

[00:12:30] Speaker 03:
And I'd like to point out... Sorry, this is where we keep... The language here is troubling to me because they've made all the data public.

[00:12:39] Speaker 03:
And then the article went through a scientific peer review process.

[00:12:44] Speaker 03:
In fact, it's been published.

[00:12:45] Speaker 03:
They have a number of publications, all of which have gone, I mean, assumption is, I might have been wrong, scientific peer review process.

[00:12:54] Speaker 03:
And there's no argument by you that they disclosed, they lost this privilege because they disclosed all these model runs as part of the scientific review process.

[00:13:08] Speaker 04:
Your honor, there's nothing in the record indicating that the 200 model runs, which they refused to disclose to us, were disclosed to the peer reviewers in the two publications that ultimately published what they call their four best modeling scenario.

[00:13:26] Speaker 03:
You have a FOIA request that says all of your model runs that have been disclosed to members of the public already.

[00:13:34] Speaker 04:
Well, we originally filed a very broad FOIA request, which would have encompassed that.

[00:13:41] Speaker 04:
And I have no reason to believe that the withheld model runs, which consists of input and output data that they have not provided, were provided to the peer reviewers.

[00:13:52] Speaker 02:
Let us hear from counsel.

[00:13:54] Speaker 02:
Whoops, sorry, Judge Wilkins.

[00:13:55] Speaker 01:
I just wanted to ask one question, Judge Rogers.

[00:14:02] Speaker 01:
if they had been provided to peer reviewers for the purpose of peer review, that wouldn't mean that they were public in the sense of they would be outside the protection of the deliberative process privilege in FOIA, right?

[00:14:23] Speaker 01:
Because

[00:14:23] Speaker 01:
the peer reviewers would have been reviewing them kind of on the same terms that someone reviews a draft article for publication with the requisite confidentiality protections that flow with that, right?

[00:14:46] Speaker 04:
Well, the peer reviewers' substantive and editorial comments on the proposed publications

[00:14:53] Speaker 04:
would be covered by the privilege.

[00:14:56] Speaker 04:
We are not seeking that.

[00:14:57] Speaker 04:
We are just seeking the data.

[00:14:59] Speaker 04:
And my understanding is that when a scientist in academia or the private sector, for example, publishes a study, they must make all study-related data available, not just to peer reviewers, but to anyone on, you know, upon request.

[00:15:18] Speaker 02:
No, but you've been making certain assumptions throughout your brief and here.

[00:15:23] Speaker 02:
Let's hear from counsel for APLE, and then we'll give you a couple of minutes.

[00:15:32] Speaker ?:
Thank you.

[00:15:32] Speaker 00:
Good morning, Your Honors.

[00:15:33] Speaker 04:
Good morning.

[00:15:33] Speaker 00:
May it please the court?

[00:15:35] Speaker 00:
I'm Josh Kopp on behalf of the government.

[00:15:38] Speaker 00:
The preliminary model runs are deliberative because they reflect agency scientists' evolving thinking about how to interpret and make sense of the raw data.

[00:15:47] Speaker 00:
The council, the pavement coatings technology council argues that the preliminary model runs our facts and the disclosure will not provide any insight into the deliberative process.

[00:15:56] Speaker 00:
That is wrong.

[00:15:58] Speaker 00:
As your honors have been discussing the council, sorry, the geological survey has already disclosed all of the facts, the raw data.

[00:16:06] Speaker 00:
Specifically PCTC has received the chemical analyses of the lake sediment samples and the pH fingerprints of the potential sources of contamination.

[00:16:15] Speaker 00:
their own scientists could use that raw data to create any model runs that they wish.

[00:16:20] Speaker 00:
What the Geological Survey has withheld are selective compilations and analyses of that raw data that reflect the process by which agency scientists interpreted and analyzed the data.

[00:16:33] Speaker 03:
Were those model runs that they want disclosed to peer reviewers?

[00:16:37] Speaker 03:
Is that part of the scientific review process?

[00:16:41] Speaker 00:
Some of them were.

[00:16:41] Speaker 00:
Some of these preliminary model runs were encompassed in preliminary drafts of the study, which were sent to reviewers within the geological survey and at the journal where the geological survey sought to publish the study.

[00:16:54] Speaker 00:
Those reviewers gave feedback.

[00:16:56] Speaker 00:
The scientists went and refined their model.

[00:16:59] Speaker 00:
So some of them were disclosed to the reviewers.

[00:17:02] Speaker 03:
Were disclosed to, I don't care about inside the US Geological, talking about outside the geological survey journal,

[00:17:11] Speaker 03:
these journals must select the people that they want to do these peer reviews.

[00:17:14] Speaker 03:
And some of those folks did get some of these model runs that they aren't getting?

[00:17:19] Speaker 00:
I believe that they did get some of them.

[00:17:22] Speaker 00:
Those peer reviewers are within the umbrella for the purpose of the deliberative process privilege.

[00:17:29] Speaker 03:
I think that's a very open question under claimant.

[00:17:32] Speaker 00:
Pardon?

[00:17:32] Speaker 03:
I think that's a very open question under claimant.

[00:17:36] Speaker 00:
I believe that in in formaldehyde Institute versus HHS this this court held that a letter from those peer reviewers long before the Supreme Court decision in claimant.

[00:17:47] Speaker 00:
That's true.

[00:17:51] Speaker 00:
The district court considered the consultant corollary.

[00:17:54] Speaker 00:
PCTC made an argument on that basis in the district court, and they have chosen not to appeal that.

[00:18:00] Speaker 00:
So they have not argued that any information was, that any privilege was waived by disclosure to the peer reviewers.

[00:18:11] Speaker 00:
So.

[00:18:11] Speaker 03:
Can I ask you what the relevant agency

[00:18:19] Speaker 03:
decision at issue here, to which this model run selection process applies.

[00:18:24] Speaker 03:
And you've described that as the decision to publish, and not just the up or down decision to publish, but what to publish.

[00:18:34] Speaker 00:
That's correct.

[00:18:34] Speaker 03:
Can you tell me who the agency decision maker was in making that comprehensive publication decision?

[00:18:42] Speaker 03:
Was it just these two scientists?

[00:18:44] Speaker 03:
Is there someone else within USGS that has to authorize

[00:18:48] Speaker 03:
had authorized them to publish in their official capacity with their official position noted in a private journal.

[00:18:54] Speaker 03:
Who's the decision maker here?

[00:18:56] Speaker 00:
Yes, I believe there are multiple levels of review, including the scientist supervisors.

[00:19:00] Speaker 00:
There is an approving official that takes a look at all publications, or multiple approving officials that take a look at all publications that the Geological Survey puts out.

[00:19:13] Speaker 03:
And were all these model runs disclosed to them?

[00:19:16] Speaker 03:
when they made these decisions about their model runs?

[00:19:20] Speaker 00:
I don't believe that all of them were disclosed to those approving officials.

[00:19:23] Speaker 00:
The approving officials may have seen some in the preliminary drafts.

[00:19:26] Speaker 00:
They certainly knew that these preliminary model runs existed, and they were relying on the feedback that the scientist peers of the Geological Survey had given to the authors.

[00:19:36] Speaker 00:
The approving officials were relying on that feedback to ensure the publication was.

[00:19:40] Speaker 03:
It's not just that somebody deliberates inside an agency.

[00:19:45] Speaker 03:
The point of the deliberation has to be to inform the ultimate decision-makers decision.

[00:19:52] Speaker 03:
So what I'm trying to find is where there's anything in this record that explains how when these two scientists were running their models in their laboratory or office or wherever they did it and making these decisions, they were doing that to inform the ultimate decision-makers

[00:20:13] Speaker 03:
determination about whether to publish or not, as opposed to they were making it for their own scientific judgments.

[00:20:22] Speaker 00:
Right, well, they were using these deliberations to ensure that the draft that they present to the approving official would be the best possible and that the models that they are seeking to publish are sufficiently reliable.

[00:20:36] Speaker 00:
And they are ensuring that through this deliberative process with their peers at USGS,

[00:20:41] Speaker 00:
they are ensuring the quality of their research.

[00:20:43] Speaker 00:
And the approving official relies on that process in determining whether to approve.

[00:20:48] Speaker 03:
It relies upon them transmitting this information about how they chose the models that they did and the ones that they didn't.

[00:20:54] Speaker 03:
And what I'm afraid of here is that this deliberative process might have been these scientists' deliberative process to inform themselves about running of the experiments.

[00:21:05] Speaker 03:
But I haven't seen a connection to how, when they reject model runs

[00:21:11] Speaker 03:
three, four, five, six, seven, eight, and nine.

[00:21:13] Speaker 03:
They did that to inform that person way down the line years later decision whether to publish or not.

[00:21:22] Speaker 03:
And you have to make that connection.

[00:21:24] Speaker 03:
Their deliberative decisions have to been to inform that person months or years later about whether to publish or not.

[00:21:32] Speaker 00:
I don't believe that there is any case that has held that the ultimate decision maker has to have seen the material for it to be covered by the deliberative process privilege.

[00:21:39] Speaker 03:
The deliberative process is to allow the agency to inform the agency decision maker's decision.

[00:21:47] Speaker 03:
That's what the deliberative process is.

[00:21:50] Speaker 03:
I mean, I worked in government and we would sit around and shoot the breeze debating issues all the time.

[00:21:55] Speaker 03:
They weren't to inform any ultimate decision made by

[00:21:59] Speaker 03:
agency.

[00:22:00] Speaker 03:
They were deliberative between us, but they weren't informing an agency decision-maker's final agency action.

[00:22:07] Speaker 03:
And I'm having a lot of trouble getting from, and this may just be my ignorance of scientific process, getting from their deliberations about their scientific study as to those decisions were being made to inform the decision-maker way down the road who decided whether to publish

[00:22:28] Speaker 00:
Well, their deliberations were certainly made in order to ensure that what they presented to the ultimate decision maker would be reliable and would gain his or her approval.

[00:22:41] Speaker 00:
If the scientists are considering a number of potential models.

[00:22:44] Speaker 03:
They don't share with that ultimate decision maker 196 no's.

[00:22:51] Speaker 03:
Then how did those decisions inform the ultimate decision maker?

[00:22:57] Speaker 00:
Well, they can assure the ultimate decision maker that the scientists have done their due diligence and have looked at a number of options to ensure that they are presenting the most reliable draft for publication.

[00:23:09] Speaker 03:
They would have decided to tweak the model here to not take that approach to adjust the data here to include this input and to exclude that input.

[00:23:21] Speaker 03:
That never gets passed down the line to the ultimate decision maker.

[00:23:24] Speaker 00:
There were certainly some preliminary model runs that could be discarded out of hand or, you know, some of these preliminary model runs were not intended as actual reliable model, but were intended to test or break the model.

[00:23:37] Speaker 00:
So for instance, scientists would use a pristine mountain lake and give the model only urban sources of PAH.

[00:23:43] Speaker 00:
And the goal was to see if the model would work in returning essentially the result that

[00:23:49] Speaker 00:
there is no result.

[00:23:50] Speaker 00:
And so, you know, some of those models were certainly not presented, you know, didn't need to be presented to the approving official, but they were absolutely essential to the scientist process of reaching the most reliable model results that they could then put in this draft publication presented to the approving official.

[00:24:09] Speaker 03:
You guys picked what the relevant agency decision here was, and you didn't pick the scientific determination

[00:24:17] Speaker 03:
by the USGS as to causal relationship between these sealants and these increased levels of this toxic chemical in the water.

[00:24:30] Speaker 03:
You have picked the decision to publish.

[00:24:35] Speaker 03:
And that's my question is I don't see in the affidavits to record here anything that says when these decision makers were doing, these scientists were doing their study,

[00:24:46] Speaker 03:
that the model runs you're not disclosing were part of, were deliberations that were undertaken to inform the ultimate decision-makers final agency action of issue.

[00:25:00] Speaker 00:
Your honor, our position is that the agency decision at issue is whether and in what form to publish and that in what form certainly includes what results to publish.

[00:25:10] Speaker 00:
So, you know, which model runs, what the ultimate conclusions are of what are the main contributors of all of these 196 model rounds with the ultimate decision.

[00:25:21] Speaker 03:
The ultimate decision makers sort of see the whole package of everything they've done.

[00:25:26] Speaker 00:
I don't believe that he saw all of the preliminary model runs.

[00:25:30] Speaker 00:
I do believe that he saw

[00:25:32] Speaker 00:
the input that the authors received from their peers, some of whom had seen these preliminary model runs, I believe that the approving official took a look at the process that the agency scientists had been through, including their formulation of some of these model runs.

[00:25:46] Speaker 01:
But we don't have any of that in the record, right?

[00:25:50] Speaker 00:
That is correct.

[00:25:51] Speaker 00:
I don't believe that

[00:25:53] Speaker 00:
what the approving officials look at is in the record.

[00:25:56] Speaker 00:
But again, I don't think it is necessary because these agency officials are acting on behalf of the agency and the deliberate process privileges, you know, covers the process of the agency, including the lower level scientists.

[00:26:07] Speaker 01:
You have the burden of establishing the privilege, right?

[00:26:12] Speaker 01:
The appellant doesn't have that burden.

[00:26:13] Speaker 01:
Plaintiffs don't have that burden.

[00:26:15] Speaker 00:
That's correct, but I don't think it's relevant what the approving officials saw or didn't see.

[00:26:20] Speaker 00:
What's relevant is that the agency officials, the agency scientists, as part of their decision-making process, were using these preliminary model runs to test out their hypotheses and to test out different approaches for analyzing the raw data.

[00:26:34] Speaker 01:
And what's in the record is that, and what you've said in your brief is that this is basically kind of how you calibrate a model, right?

[00:26:43] Speaker 01:
This is standard.

[00:26:45] Speaker 01:
this is something that's done in any instance where you're trying to develop a model to replicate or to, you know, try to replicate and answer a question, right?

[00:26:59] Speaker 00:
Running preliminary model runs, I believe, is quite standard.

[00:27:02] Speaker 01:
If that's the case, then where is there evidence in the record that if we disclose these model runs or we order their disclosure,

[00:27:14] Speaker 01:
that it will discourage them from being done in the future or that it will somehow harm the quality of future agency decision-making.

[00:27:26] Speaker 00:
So which models the scientists run is an extremely subjective, it is an exercise of subjective, quantitative and qualitative judgment.

[00:27:38] Speaker 00:
And we can certainly, I believe that

[00:27:40] Speaker 00:
If the agency scientists knew that their preliminary model runs were going to be disclosed, they would certainly be less candid, you know, for fear of being ridiculed.

[00:27:50] Speaker 00:
That's not an affidavit.

[00:27:52] Speaker 01:
That's not in any of the declarations.

[00:27:55] Speaker 00:
I believe that Dr. Van Meter and Dr. Van Muller, I believe that they both say that they would be discouraged from running some of these preliminary models if they knew that they... I did not see that.

[00:28:08] Speaker 01:
I'd like for you to find that language because I looked for it and I didn't see it.

[00:28:13] Speaker 00:
Yes, I can take a quick look, Judge.

[00:28:19] Speaker 01:
I think that the best that they say, if you look at JA-27, Dr. Van Meters, is that it will give outsiders an opportunity to confuse the public and discredit our work.

[00:28:34] Speaker 01:
And so they say that they don't, you know, they think that's something bad that will happen, but they never says that in the future, then I'll do preliminary model runs differently, or I'll be inclined not to do them.

[00:28:52] Speaker 01:
And there's similar language in the declaration from Dr. Mahler,

[00:29:01] Speaker 01:
But I don't see anything that says that disclosure of this is going to kind of change the way that we do our business.

[00:29:14] Speaker 00:
I'm not sure if they explicitly say that it would change the way they do their business.

[00:29:17] Speaker 00:
They do say, as you mentioned, that it would confuse the public.

[00:29:21] Speaker 00:
And I believe, first of all, that is its own interest protected by the deliberative process privilege, public confusion.

[00:29:28] Speaker 00:
But I think that we can certainly imagine

[00:29:32] Speaker 00:
We took another context, for example, climate change.

[00:29:35] Speaker 00:
If agency officials knew that climate, that, you know, their climate change research, all their preliminary models would be disclosed to the public, agency officials might pressure lower level scientists to only run certain models that are consistent with what those agency officials want to ultimately publish.

[00:29:50] Speaker 00:
Or the scientists themselves may feel inhibited from running certain models if they are afraid that those models will show something other than what they think their ultimate conclusion will be.

[00:29:59] Speaker 03:
Because those models are not done for publishing articles in journals.

[00:30:01] Speaker 03:
Those are done for deciding what the final agency position or agency action will be in a regulation or a policy of some sort.

[00:30:13] Speaker 03:
The difference here is it seems to me you've picked an unusual agency action, and that is to publish an article in official capacity.

[00:30:24] Speaker 03:
And what I'm having trouble with is that you seem to be conflating

[00:30:30] Speaker 03:
the decision to come to a reliable scientific result through this trial.

[00:30:38] Speaker 03:
And the scientists' own decision and concern about doing that with the decision of someone else who you said there are many layers of review.

[00:30:45] Speaker 03:
So I don't know how many layers this is, but apparently there's somebody at the end who signs off and says, yes, we will publish this in the name of the US Geological Survey, or at least

[00:30:56] Speaker 03:
they can use their official titles.

[00:30:58] Speaker 03:
Does that mean, I guess I should clarify one thing, does that mean it's a United States Geological Survey position when scientists publish in a journal in their official capacity?

[00:31:10] Speaker 00:
Yes, I believe that's correct.

[00:31:12] Speaker 03:
We believe that's correct, but we don't even know that, right?

[00:31:15] Speaker 03:
So we're not even sure if that's what's going on.

[00:31:18] Speaker 03:
But I will assume that for now because they get to use their official

[00:31:22] Speaker 03:
But that just seems to be an awful lot that's happened in between.

[00:31:25] Speaker 03:
I'm just not convinced every time they're standing there a year or so before and they run a model run and they go, that's crazy.

[00:31:35] Speaker 03:
But it was part of the decision whether to tell someone way down the line, this person, whether she should authorize them to publish in their official capacity.

[00:31:43] Speaker 03:
And that's the connection I think we need for deliberative process.

[00:31:47] Speaker 00:
I think this is no different than those draft agency histories that were at issue in Dudman and Russell.

[00:31:53] Speaker 00:
In those cases, the court wasn't concerned whether the ultimate decision maker had viewed the preliminary draft.

[00:31:59] Speaker 00:
If there was a preliminary draft that got changed somewhere along the process, somewhere on its way up to the official in the Air Force that was going to ultimately approve whether these draft histories were made public, that was enough.

[00:32:13] Speaker 00:
because it exposes the deliberative process of the primary author, or really of anyone inside of the agency that's helping to formulate the history at issue.

[00:32:22] Speaker 00:
And so, you know, those cases also, of course, involve publications rather than policies.

[00:32:30] Speaker 03:
Maybe I just don't know how these folks work.

[00:32:35] Speaker 03:
When these scientists started this process, was their goal to write an article, or was their goal to make a scientific determination about the relationship between

[00:32:43] Speaker 03:
this pavement coating and toxins in water?

[00:32:50] Speaker 00:
My guess is that I think that it was both.

[00:32:52] Speaker 00:
They expected to make a determination of the relationship and to publish those findings for the benefit of the scientific community.

[00:33:03] Speaker 00:
Because the geological survey doesn't make policy or.

[00:33:09] Speaker 03:
We don't have them saying what the point

[00:33:13] Speaker 03:
Their deliberations, as they describe them, seem to be to get this science right, to figure out this question.

[00:33:18] Speaker 03:
That's what they're deliberating about.

[00:33:21] Speaker 00:
That's right.

[00:33:21] Speaker 00:
But also to publish it for the benefit of the scientific community and the benefit of the policymakers.

[00:33:29] Speaker 03:
Do the declarants here, the two scientists that did this study, do they state their declarations?

[00:33:33] Speaker 03:
Because I didn't see this either.

[00:33:36] Speaker 03:
We were making this deliberate decision, deliberative decisions to inform the ultimate decision maker on whether or not to publish this in the name of the USGS.

[00:33:46] Speaker 00:
I'm sorry.

[00:33:47] Speaker 00:
I can you repeat the question?

[00:33:49] Speaker 03:
Declarations.

[00:33:49] Speaker 03:
They say when we were making our decisions about these 196 model rounds, we rejected or the four that we thought were best and why we thought they were missed.

[00:33:59] Speaker 03:
They were making that because we wanted to be able to inform the ultimate decision maker about whether to publish.

[00:34:06] Speaker 00:
They don't say exactly that.

[00:34:07] Speaker 00:
They say that these preliminary model runs were an essential part of them forming their ultimate conclusion, which was then approved by the approving official published.

[00:34:18] Speaker 00:
Pardon.

[00:34:19] Speaker 03:
The scientists ultimate conclusion was a scientific relationship.

[00:34:25] Speaker 00:
That's right.

[00:34:30] Speaker 00:
This is no different, I believe, than a professor workshopping a paper at a university.

[00:34:35] Speaker 00:
He receives feedback from his or her peers and then goes back and refines the paper.

[00:34:40] Speaker 00:
And that's exactly what is going on here.

[00:34:42] Speaker 03:
The scientists... When they went to these outside journals, this wasn't a government journal that was published.

[00:34:48] Speaker 03:
It was an outside journal.

[00:34:49] Speaker 03:
And then did the government pick the peer reviewers or did the outside journal?

[00:34:54] Speaker 00:
The outside journal.

[00:34:55] Speaker 03:
The outside journal.

[00:34:55] Speaker 03:
And did the outside journal pick those peer reviewers for the purpose of

[00:35:00] Speaker 03:
telling the government what the best science is, whether these meet the scientific standard, or for telling the journal whether these meet proper scientific standards to be published.

[00:35:14] Speaker 00:
I believe the intent of the – I can't speak to the intent of the journal, but I believe that it is to inform the journal whether the

[00:35:20] Speaker 00:
papers worthy of publication, but also to help the study authors get the publication into shape.

[00:35:25] Speaker 00:
Because the reviewers don't give a yes or no, up or down.

[00:35:29] Speaker 00:
They give feedback to the authors who then go and modify the paper, edit, run new models.

[00:35:37] Speaker 03:
Notify it because it's not getting published unless they clean something up.

[00:35:40] Speaker 03:
But I assume the point of it is to have comments so that the journal knows whether this is something that meets the journal standards for publication.

[00:35:51] Speaker 00:
I think that is one of the purposes, but again, I do believe that part of the, you know, the journal is also, doesn't select reviewers only for its own benefit of giving a yes or no, but also to contribute to the scientific discussion and to aid the authors in improving their research.

[00:36:12] Speaker 00:
Unless there are further questions I have.

[00:36:13] Speaker 02:
You're describing this process, just so let me be clear.

[00:36:17] Speaker 02:
It's not, it's not two separate parts.

[00:36:21] Speaker 02:
but rather it's just one continuous effort until a decision is ultimately made to publish.

[00:36:39] Speaker 00:
I'm sorry, I'm not sure which two parts your honor is referring to.

[00:36:41] Speaker 02:
I'm just trying to understand.

[00:36:43] Speaker 02:
You have these two scientists doing all of this work and then they decide

[00:36:53] Speaker 02:
that the the best evidence they have their view among the models are the four that they disclose and then they discuss what they found.

[00:37:14] Speaker 02:
So now here the appellant is seeking not simply the raw data but

[00:37:23] Speaker 02:
it wants the worked up, it wants to understand the background to the workup of the document that was ultimately published.

[00:37:37] Speaker 02:
And as I understand it, anything that went into that, that the parties, the scientists, the head of the survey didn't agree to publish

[00:37:54] Speaker 02:
is simply unavailable.

[00:37:57] Speaker 02:
You have the raw data, you have the published study with the four.

[00:38:05] Speaker 02:
And in that sense, there's a deliberation going on as far as the scientists are concerned.

[00:38:16] Speaker 00:
That's right.

[00:38:17] Speaker 00:
What PCTC seeks are the approaches to or the interpretations of data that the scientists ultimately decided were not those that were the most reliable and worthy of publication.

[00:38:29] Speaker 00:
So using the raw data, PCTC scientists could run these exact same models.

[00:38:34] Speaker 00:
What they wouldn't know is whether that model is one that the USGS scientists considered before settling on therefore

[00:38:44] Speaker 00:
final models that were worthy of publication.

[00:38:46] Speaker 00:
So what they seek is insight into the deliberative process.

[00:38:51] Speaker 00:
I mean, exactly.

[00:38:52] Speaker 00:
They have the raw data.

[00:38:54] Speaker 00:
They have the results.

[00:38:55] Speaker 00:
What they don't know is what approach the scientists use to interpret that data on the path from getting from raw data to ultimate publication.

[00:39:13] Speaker 00:
Unless there are further questions, we ask that your honors affirm the judgment of the district court.

[00:39:17] Speaker 02:
All right.

[00:39:18] Speaker 02:
Council for appellant.

[00:39:29] Speaker 04:
Yes, thank you.

[00:39:30] Speaker 04:
Well, again, we're not seeking interpretations.

[00:39:32] Speaker 04:
We are only seeking the 200 modeling runs, and we heard

[00:39:37] Speaker 04:
government council just explained that with the tens of thousands of pages of raw laboratory data that they have provided, we can try to somehow do our own modeling runs with all these limitless possible combinations of input variables and try to guess what their 200 modeling runs were.

[00:39:58] Speaker 04:
I'd like to refer the court to the declaration of USGS FOIA Officer Brian May.

[00:40:05] Speaker 04:
He says in his declaration, first of all, that the withheld model runs were not used in the final decision.

[00:40:13] Speaker 03:
What page of the day are you on, please?

[00:40:15] Speaker 04:
That would be in the appendix on page 49.

[00:40:19] Speaker 04:
He says that the withheld modeling data were not used in the final publication, were not used in the, quote, final decision.

[00:40:29] Speaker 04:
He also says on page 53 of the appendix

[00:40:34] Speaker 04:
that it was the USGS approving officials, those are the people who ultimately approved publication.

[00:40:43] Speaker 04:
It is the USGS approving officials, quote, internal discussions and, quote, candid feedback to the study authors that, quote, comprised the USGS deliver their process.

[00:40:57] Speaker 04:
We're not asking for any documents reflecting those internal discussions.

[00:41:01] Speaker 04:
We're not asking for any documents reflecting the candid feedback that the approving officials gave to the study authors.

[00:41:09] Speaker 04:
We're not asking for any deliberative documents.

[00:41:12] Speaker 04:
We just want the modeling run, which we've described as the input and output data.

[00:41:18] Speaker 04:
And the other point I'd like to make.

[00:41:22] Speaker 02:
My question then is, I know you characterize it that way, but is that an accurate way to characterize it?

[00:41:30] Speaker 04:
It is.

[00:41:31] Speaker 02:
The scientist takes the raw data and does something with it, right?

[00:41:38] Speaker 04:
Yes, I think the answer to your question, Judge Rogers, is set forth in the declaration of Dr. Van Meter, who is one of the two study authors.

[00:41:47] Speaker 04:
He explains in detail

[00:41:50] Speaker 04:
not only how they selected a particular computer model, but also how they formulated a modeling run.

[00:41:58] Speaker 04:
He explains that the modeling run includes the raw laboratory data, the sediment readings and the PAH profiles, the sort of more than a dozen different types of PAHs.

[00:42:10] Speaker 04:
Then he talks about the modeling parameters

[00:42:13] Speaker 04:
And he explained, it's kind of like turning the knobs on a stereo hi-fi, there's all different ways to do it.

[00:42:22] Speaker 04:
And he says there are so many possible combinations of inputs and modeling parameters, the possible outputs for a given model are virtually limitless.

[00:42:34] Speaker 04:
And the other point I'd like to make in rebuttal.

[00:42:38] Speaker 02:
Well, let's finish up on that.

[00:42:40] Speaker 02:
He's describing the intellectual

[00:42:45] Speaker 02:
element that goes in between the raw data and I'll call a draft.

[00:42:58] Speaker 02:
And then it goes through some refining processes, but it's still the scientists input.

[00:43:06] Speaker 04:
That's correct, Judge Rogers.

[00:43:09] Speaker 04:
And what you have just described is the scientific method, a method that Mr. Koppel said was standard running these computer modeling runs.

[00:43:17] Speaker 04:
And I think the question for the court is whether the court would like to take the unprecedented step of stretching

[00:43:26] Speaker 04:
exemption five to encompass the entire scientific method.

[00:43:30] Speaker 04:
And if the court were to do that in this case, it would set a precedent for federal government scientists in all agencies that employ federal government scientists.

[00:43:41] Speaker 04:
And any federal government scientist, just by tagging modeling data as exploratory analysis and saying, well, it was my scientific deliberations,

[00:43:53] Speaker 01:
we'd be able to hide the ball, which is exactly- You're saying that that would be, we would be taking an unprecedented step, but I don't recall seeing you cite anything that says, precedent-wise, that says that the scientific method is outside of the deliberative process privilege.

[00:44:15] Speaker 04:
Well, I think one case we've cited, for example, is the American Radio Relay League case.

[00:44:22] Speaker 02:
I'm not familiar with that case, as you know, and that's a different case.

[00:44:27] Speaker 02:
There was a study, and the agency had to release all of it.

[00:44:34] Speaker 02:
Now, we're between the raw data and the ultimate study.

[00:44:41] Speaker 02:
American Radio set the study.

[00:44:45] Speaker 02:
Now, you're asking for something that happened in between.

[00:44:51] Speaker 02:
And the question is whether or not that comes within exemption five or not.

[00:44:56] Speaker 04:
No, our request refers to the urban lakes study, just like there was a study involved in the American radio relay case.

[00:45:03] Speaker 04:
And what, at my understanding from reading that case, is what the FCC was trying to do in that case was to hide, and that's the term, that's the word used in the decision, hide parts of the study.

[00:45:18] Speaker 02:
I'm asking you, is there something between the raw data

[00:45:24] Speaker 02:
And the published report.

[00:45:27] Speaker 04:
Yes.

[00:45:28] Speaker 02:
What is it?

[00:45:29] Speaker 04:
Modeling runs.

[00:45:32] Speaker 02:
But that's not a study, is what I'm trying to get at.

[00:45:35] Speaker 04:
It's an inherent, it's an intrinsic part of the study.

[00:45:40] Speaker 04:
The word study.

[00:45:42] Speaker 04:
Okay.

[00:45:43] Speaker 04:
The word study can be the scientific method or it could be the publication of the study.

[00:45:48] Speaker 04:
And what we have here, you know, is a scientific process that the study authors conducted running more than 200 modeling runs that culminated in a proposed publication, which, according to their words,

[00:46:05] Speaker 04:
only have disclosed the modeling runs for what they consider the four best modeling scenarios.

[00:46:11] Speaker 04:
Modeling scenarios that say basically that refined tar sealant is the primary source of PAHs in urban lakes, and they refuse to disclose the other models, as Your Honor has pointed out.

[00:46:27] Speaker 04:
And what counsel to the government is- Go ahead, excuse me.

[00:46:31] Speaker 04:
What Council for the Government is doing, respectively, is conflating a scientist's deliberative process, where he began his argument by referring to scientists' thinking, that's also the term used in their brief, conflating that with the agency's deliberative process on whether to publish a study that is presented to them by agency scientists.

[00:46:56] Speaker 04:
And again, FOIA Officer May

[00:46:58] Speaker 04:
says that the modeling runs that were withheld were not used in that final agency decision.

[00:47:06] Speaker 04:
He says what was the USGS deliver the process were the internal discussions among the USGS approving officials and the candid feedback they gave to the study authors.

[00:47:18] Speaker 04:
We don't know what that feedback was.

[00:47:20] Speaker 04:
We're not asking for that in this appeal.

[00:47:22] Speaker 04:
We just want the modeling runs, respectively.

[00:47:25] Speaker 04:
And one last point, if I may, very quickly.

[00:47:28] Speaker 04:
This is a very influential study.

[00:47:30] Speaker 04:
As we point out in our brief, it, along with the House does study, which we haven't had time to discuss here, but is discussed in our brief, have led many state and local governments to ban this product.

[00:47:44] Speaker 04:
And although USGS doesn't set policy, even a casual look at their refined tar sealant webpage indicates that they rely on these studies to essentially recommend that state and local governments ban the product.

[00:48:03] Speaker 04:
So we have the agency promoting a published study and hiding the ball

[00:48:10] Speaker 04:
the ball being what we need in order to test the study's conclusions.

[00:48:16] Speaker 02:
I want to push you just half a minute.

[00:48:19] Speaker 02:
The two scientists say we ran over 100 models.

[00:48:24] Speaker 02:
We picked the best four.

[00:48:27] Speaker 02:
Those are the four that we think best support our conclusions.

[00:48:34] Speaker 04:
Yes.

[00:48:36] Speaker 04:
That's what they're saying.

[00:48:37] Speaker 04:
And we wonder whether what they consider their best scenarios are the scenarios that essentially condemn this product or find tar seal it.

[00:48:50] Speaker 02:
They're not worthy of any credence.

[00:48:55] Speaker 04:
Well, I'm not going to say that they're not worthy of credence, but I'm sorry, Your Honor.

[00:49:00] Speaker 02:
They say these are the best.

[00:49:03] Speaker 02:
And the argument is,

[00:49:07] Speaker 02:
Maybe they're not the best.

[00:49:09] Speaker 02:
Maybe these scientists are no good.

[00:49:12] Speaker 02:
Well, we're certainly not going to say that, Your Honor, but what they... I just want to understand what it is that we're after.

[00:49:21] Speaker 04:
That's all I... What we are after, Your Honor, is the ability to see how the outcomes of those other 200 modeling runs differ from the outcomes of these four scenarios.

[00:49:37] Speaker 02:
And so as Judge Millett asked you at the beginning, your own scientists could do all this, and presumably that most scientists who would approach this question would approach it in similar ways.

[00:49:53] Speaker 04:
We can only do that, Your Honor, with the 200 modeling runs.

[00:49:58] Speaker 02:
No, that's one way you could do it.

[00:50:01] Speaker 04:
Pardon, Your Honor?

[00:50:02] Speaker 02:
That's one way you could do it.

[00:50:04] Speaker 04:
I believe it's the only way.

[00:50:05] Speaker 02:
Oh, it's not the only way.

[00:50:08] Speaker 02:
You can do 1,000 models.

[00:50:12] Speaker 02:
I mean, I realize we're talking about money here, all right?

[00:50:17] Speaker 02:
And setting up a whole chemistry lab.

[00:50:19] Speaker 04:
Yes, I mean, yes.

[00:50:23] Speaker 02:
I just want to understand what it is we think that in a normal situation, the deliberative

[00:50:34] Speaker 02:
process would cover.

[00:50:40] Speaker 02:
You're saying all the work you do, and you may be right, all the work that is done, reaching your conclusion means that if you're on the government payroll, you've got to turn it over.

[00:51:00] Speaker 04:
And if you work for a corporation or for a university, there'd be no question that you would have to make these 200 modeling runs available.

[00:51:13] Speaker 04:
But USGS wants its scientists to have a privilege, if you will, that no other scientists have.

[00:51:22] Speaker 04:
And again, what do we do with tens of thousands of pages of raw laboratory data?

[00:51:32] Speaker 04:
Yes, you're right, Your Honor, but we could come up with some modeling runs.

[00:51:37] Speaker 04:
But we don't know whether they would be the 200 modeling runs that the study authors ran.

[00:51:44] Speaker 01:
So let's suppose that the agency that we were dealing with was an agency like the EPA.

[00:51:53] Speaker 01:
And what the scientists had done was developed a model that was used to ultimately promulgate a regulation by the EPA to regulate or restrict or ban in some way these pavement coatings.

[00:52:15] Speaker 01:
And you submitted a FOIA request.

[00:52:20] Speaker 01:
So there's kind of would the model runs be covered by the deliberative process privilege there?

[00:52:34] Speaker 04:
No, they would not.

[00:52:36] Speaker 04:
Because the model runs are simply factual data.

[00:52:39] Speaker 04:
What are the inputs?

[00:52:40] Speaker 04:
What are the outputs?

[00:52:42] Speaker 01:
And so it doesn't turn on whether the agency is a

[00:52:47] Speaker 01:
policy making agency or not.

[00:52:49] Speaker 01:
You're just saying that this is science and so deliberative process of these scientists is not covered by the privilege or you're saying that the model runs themselves don't reveal the deliberative process.

[00:53:13] Speaker 01:
I'm trying to understand.

[00:53:15] Speaker 04:
I'm saying that the former, your honor, that the scientists thinking is not covered by the deliver the process privilege.

[00:53:22] Speaker 04:
And to use the words again, we cite a number of cases, but I think American radio relay is close.

[00:53:29] Speaker 04:
It says, quote, purely factual investigative matters, close quote, are not protected.

[00:53:36] Speaker 04:
the privileges designed to protect processes by which agencies' policies are formulated and recommended.

[00:53:43] Speaker 04:
Now, USGS doesn't establish policies, but it's done the next best thing.

[00:53:48] Speaker 01:
Well, we were talking about my EPA hypothetical, and

[00:53:58] Speaker 01:
I mean, doesn't the doesn't your own briefing.

[00:54:02] Speaker 01:
I mean, part of your own briefing is, is that the data and is not helpful because we need to know kind of like which data they selected in in in in which models they ran so

[00:54:20] Speaker 01:
disclosing that discloses their thinking.

[00:54:23] Speaker 01:
This isn't just data, it's disclosing choices that those scientists made, right?

[00:54:29] Speaker 04:
That is exactly correct, but we submit that that type of scientific thinking is not the type of deliberation that is intended to be covered

[00:54:41] Speaker 04:
by a narrow privilege that goes to the formulation and recommendation of agency policies and the result of the rule or the interpretation that the government is advocating is to envelop the entire scientific method, all that scientific thinking at EPA and every other... What is your best case or cases for that proposition?

[00:55:09] Speaker 01:
American radio relay?

[00:55:11] Speaker 04:
I think that's one very good case, yes.

[00:55:16] Speaker 04:
National Home Builders is another one.

[00:55:20] Speaker 01:
All right.

[00:55:20] Speaker 01:
Thank you.

[00:55:22] Speaker 04:
Thank you.

[00:55:23] Speaker 02:
We'll take the case under advisement.