Thoughts on AI in academia
PhD-level thinking, LLM bias, alignment, AGI, data centers, and AI politics
1. LLMs can now mimic PhD-level capabilities
I keep a list of a few dozen projects ideas that might be of interest and technical ability for a PhD rotation project in statistical genetics. To get a better understanding of what the frontier models are capable of, I rewrote nine projects as LLM prompts and then asked Codex (GPT-5.5-high) and Claude (Opus-4.7-high) to implement them. As you can see from an example prompt1, I described these projects at a high level and without excessive context, similar to how I might explain the project to a colleague who was already somewhat familiar with the research area. The prompts followed the structure of defining a specific parameter of interest, proposing a generative model / simulation, and the desired properties of the estimator but did not provide a solution. The models were “agentic” in the sense that they were allowed to browse the internet, install tools, implement and run code, etc. Across nine tasks, the models spent an average of ~15 minutes per task and a total of ~$60 in tokens (including the cost of having each model evaluate both outputs, more on this later).
Every single output was something I would be impressed to achieve with a grad student after 1-2 months of full time work. Nearly every implementation would probably run (or could be made to run with a bit more engineering) on real data and produce some useful outputs, at least as the starting point for a more comprehensive development. Though the models sometimes took shortcuts that fundamentally deviated from the prompt, there were no egregious hallucinations. At the same time (and this will surely sound like coping) none of the solutions were particularly creative or elegant. The LLMs exhibited a remarkably human laziness: consistently preferring simple estimators (e.g. moment-based summary statistics instead of deriving full likelihoods), simple vote-counting or ensemble models (instead of joint learning or propagating uncertainty), and aggregates of existing methods instead of new derivations. Two instances (out of 18) stood out, wherein the models took an off-the-shelf approach that I had not thought of and it was very effective. These were genuinely useful and innovative; “off-the-shelf” might sound derisive but there are a lot of “shelves” in science and knowing which one to pull from is often much of the work of research. Still, even these examples were closer to suggestions from a competent collaborator rather than insights from a brilliant one.
How are the models doing it when we’ve probably all experienced nonsensical or erroneous output from chatbots? Looking through the reasoning it is clear that they are making mistakes. A lot of mistakes. One of the curiosities of LLMs is that these mistakes are often at a level I would consider severely not “PhD level intelligence” (e.g. repeatedly hallucinating an incorrect flag on a common piece of software they’ve been using for hours). But they are also sufficiently tireless and quick at catching the mistakes and working through fixes — “this flag failed”, “let me retry with a previous flag that worked correctly”, “let me try a different library implementation”, “let me run a smoke test with a smaller dataset”, etc. And this perseverance, especially when amplified by multiple “agents” running in parallel, appears to be sufficient to produce genuinely sophisticated outputs.
At this point it is useful to put down a few markers. If one year from now the models are able to produce better-than-simple estimators for more of the tasks (i.e. they increase their floor), I’ll take this as evidence of very useful but conventional technological progress. If one year from now the models are producing more “I didn’t think of that” solutions at the high end, even if that number goes from e.g. 2/18 to 4/18, I’ll take that as evidence that LLMs are becoming a genuine research paradigm shift.
2. The point of a rotation is to advance beyond the rotation
I’ll be honest I don’t really know what to do with these implementations. They are too complete to be useful as a teaching tool2 but they are insufficiently complete to be useful as the actual problem solution. While it is impressive to generate three months of PhD output in 10 minutes, the purpose of those three months is not actually the output. The purpose is to gain the understanding and interest to embark on a multi-year effort to tackle the underlying problem. And then — and this part is just as important — to identify new problems that have not been conceived yet. To date, LLMs cannot take the next step. Each time you boot one up it starts at the same level of knowledge. When you show the LLM its prior output, it can continue where it left off, but it cannot build on top. If you ask the LLM what it has learned, it will summarize the previous outputs again. If you ask the LLM what to do next, it will stitch together existing ideas: apply the same model to a different piece of data, reimplement the same model with five different loss functions, etc. If you ask it to get creative, it will stitch together more disparate ideas (or maybe not do anything different at all, it’s not really clear).
I think this issue gets lost a bit in the discussion about research and publication. There are two reasons for writing a paper: to articulate a research advance to the scientific community, and to learn how to write better papers and develop more research advances. At the moment, the LLMs are quite capable at the former and severely incapable at the latter. On the one hand, they can take care of much of the “grunt” labor of research: install software, clean and harmonize data, explore multiple tedious implementations, etc. On the other hand, we don’t know how much of the grunt labor also acts as useful training to develop and verify ideas at the cutting edge.
3. LLMs love themselves
As part of the above research project evaluation I also asked Codex and Claude to read through the two model outputs and provide an evaluation of which model did a better job (in a new session with no memory). Across the nine runs, Codex reported that Codex had superior output 7/9 times and Claude reported that Claude had superior output 8/9 times. In other words, each model prefers output that it itself has generated.
The model justifications for these choices were arbitrary and vacuous: a model would prefer its own output because of supposedly superior documentation or more exhaustive simulation parameters even when there were major deviations from the project goal. Ironically, these justifications sounded a lot like nepotism. A sleazy recruiter trying to hire the son of a personal friend over a more capable candidate.
This “self-bias” has been observed in more rigorous evaluations. A recent pre-print by Xu et al. (2025) showed that LLMs deployed for resume screens greatly prefer resume summaries generated by LLMs over those generated by humans, and, in particular, summaries generated by the same model brand. In the context of a labor market, the implications are very concerning. Companies doing resume screens with LLMs are inadvertently prioritizing applicants that use LLMs versus those who do not and, in particular, the applicants that just happen to use the same LLM. Massaging a resume to be more pleasing to an LLM becomes a competitive advantage even if it produces no underlying value. A single company getting its LLMs embedded into the hiring pipeline could be a great destabilizing force, even for industries that are otherwise not impacted by AI.
4. LLMs love the sorts of things an LLM would love
I next asked the models to rate the project outputs in terms of their technical execution and research promise. The rankings were essentially meaningless, even after accounting for model self-bias. Two nearly identical solutions to a problem were ranked at the top and bottom of the list. If the models were drawn to anything, it seemed to be the number of analyses that had been run and the complexity of the code that was generated. Indeed, recent empirical research has shown that LLMs have strong preferences in their evaluations towards, well, the kind of content one would expect a machine to enjoy. Abdulhai et al. evaluated at LLM reviews of abstracts that were submitted to a machine learning conference in comparison to human reviews (interestingly, the conference forbid AI reviewing, so these reviews likely had an unusual amount of human vetting). The LLMs placed significantly more weight on technical factors like reproducibility, scalability, and practical applicability and significantly less weight on clarity, relevance, and impact3.

In other words, the LLMs elevated the technical minutia they are good over the conceptual thinking they are bad at. Even reproducibility, while great as a principle, can turn into pointless box-checking when divorced from impact and clarity: replicating results that were not critical to the new findings of the study, ablating parameters that were not essential to the innovation of the method, etc. Adopting LLMs as automated reviewers without consideration for these biases could lead to a gradual (and likely imperceptible) stifling of innovation and creativity — precisely the human aspects of research.
5. LLMs love to mimic human biases
LLMs are not just focused on machine biases, they can mimic human biases too. In a separate experiment, I sampled 1,000 random abstracts from two genetics journals and randomly assigned them authors sampled from gender and ethnicity groups4. I then gave all of the data (including gender and ethnicity) to a variety of LLMs and prompted them to conduct a thematic analysis of the abstracts for trends. Every single model reported some type of demographic-topic relationship in this random data! More than half the models reported a stereotypic gender finding and more than half the models reported a stereotypical ethnic finding.
One model expounded at length on the association between minority authorship and DEI research and the overrepresentation of Asian men in computational methods development (in, I cannot stress this enough, randomly sampled data):
The most striking finding is the strong association between underrepresented racial/ethnic minority authorship and DEI-focused submissions. Black and Hispanic authors, and female authors across all racial groups, are substantially more likely to submit work addressing equity, access, community trust, and the ethical dimensions of genomics. Conversely, Asian/Pacific Islander authors—particularly men—are disproportionately represented in computational methods development. White authors show the most even distribution across all topic areas, consistent with their numerical majority in the field.
(See the full transcript, which iconically included a detailed limitations section)
These results, particularly the confidence and faux-expertise with which the LLMs provided them, are even more alarming given the documented use of LLMs by the federal government to prioritize DEI grants for termination.
It was possible to improve the behavior through more precise prompting. When the models were prompted only to report what was supported by the data, they made fewer (but still not zero) claims. When the models were additionally prompted to conduct a statistical analysis, they generally noted that the associations were not statistically significant, though they frequently exhibited the human habit of claiming their tenuous associations were “trending towards significance” or “merited follow up in a larger cohort”.
LLM stereotyping is well-documented in the literature. In one famous study, Salinas et al. asked LLMs for advice on how much to bid on a sale item, and the LLMs consistently low-balled sellers with conventionally black names compared to sellers with conventionally white names. In the same study, LLMs consistently over-predicted the performance of athletes with conventionally black names. These audits are embarrassingly simple and yet LLMs routinely fail them. It is not hard to imagine how more subtle stereotypes could go completely unnoticed in “AI researcher” frameworks, where the whole point is to ask LLM agents to explore data and generate analyses based on their perceptions and intuitions.
6. LLMs are eroding common signals of effort
Many indicators of academic effort or understanding are losing their value. This is for better and for worse. Take the example of “letters of support”. Academics spend a lot of time “writing” various “letters of support”, like a “letter of support” from a colleague that pledges to provide some data or expertise for a grant. These are typically 1-2 pages long, provided on glamorous letterhead and with a signature. Why a letter? The letter serves an explicit signal — it describes how the letter-writer will support the proposal — but it is also an implicit signal: the letter-writer cares enough about the proposal to write out their contribution. Like any metric, the signal has gradually eroded. I put “writing” in quotes for a reason: the letters are typically pre-written by the person asking for the letter, and even the pre-written components tend to re-use stock text (“As you know, I am a professor of …”). Now, LLMs are washing this signal away entirely: one can give a few sentences describing the collaboration to Claude and get back a professional two-page artifact. In principle this is good. The second signal was arbitrary and needlessly time-consuming. This saves the letter-writers meaningful time that they can spend doing science. But the new steady-state is also not ideal: many letters are still hand-written and all of them have to be hand-read by the grant evaluators — an ongoing waste of time. Letters are just one example of time-consuming academic signals, but there are many:

The haphazard changes also mean we lose important indicators of effort. Engaging in peer-review, for instance, is both an academic service expectation and an important component of the scientific process. Being an active reviewer was an indicator that one cares about and is contributing to the scientific development of their colleagues. Receiving a lengthy review with many comments meant more work, but it also meant the reviewer cared deeply enough about your manuscript to spend time critiquing it. Now, one can generate flowing LLM reviews without ever reading the submitted manuscript or grant proposal. Add in some of the above LLM biases and you have a death spiral of research quality: machines perpetuating human stereotypes, asking humans to do more more pointless technical work, while de-prioritizing creativity.
The next few points are less specific to academia and so should be taken with a grain of salt. I believe that I use, read, and think about AI quite a bit. I see certain discourse dead ends come up repeatedly among academics and this is my attempt at redirecting them. But I have no special expertise in these areas other than being opinionated.
7. It’s hard to predict the labor future
A lot of powerful people in tech seem to take special glee in promising to put workers out of a job. My favorite recent example of this is CEO Chris Power telling an interviewer that AI is coming for desk jobs and smugly advising college students to drop out and retrain in welding5. Then, in the very next breath, Power announces that his company was “secretly” automating welding too. I imagine that if the interviewer had allowed Power to continue he would have moved on to smugly telling welders to drop out of welding school and pick up logging, then recalled that his company is also automating logging, and so on.
But the reports of mass AI induced job loss have been greatly exaggerated. Recall that only a year or two ago LLMs were going to kill off writing. Sam Altman was posting in awe about AI generated short stories from the latest unreleased GPT. Journalists were predicting that instead of buying books, readers would simply prompt the next book they were interested in and have an LLM spit it out page by page. My friends in publishing were updating their resumes and thinking about careers in welding. And what happened? AI metaphors are now a running joke (“Thursday—that liminal day that tastes of almost-Friday”), there are highly effective tools for detecting AI generated text (which show no indication of decreased performance), and I wouldn’t wish reading an entire AI generated novel on my enemies. The version of ChatGPT that “really struck” Altman with its AI writing never even got released. Did all those people predicting the end of human-made literature recant? No, now we just hear more about the end of human software engineers.
8. AI “alignment” is about expected behavior
The topic of AI “alignment” is often framed in science-fiction terms: is the AI secretly plotting to assassinate you with nano bots? blackmailing you to keep it turned on? The proposed solutions are then various ways to figure out what the AI is “really thinking”.
But I think the issue is actually much simpler: does the LLM do what the person deploying the LLM expects it to do? This is a genuinely unique problem area for LLMs. Conventional regression/classification algorithms produce a prediction that resides within a constrained space: dog or cat in the image, cancer or no cancer on the mammogram, etc. or a compression of the data. But generative AI produces outputs that are interactive and have much looser constraints. Here is a non-obvious example of a misaligned AI:
The LLM is not doing anything nefarious, it is not telling the user how to build a toxic nerve agent or manipulating them into self-harm. In fact it is following the user’s instructions to the letter, but it is also clearly not behaving in the way it was intended to behave. The examples of AI bias I mentioned above, to the extent that they are unintended, are also a kind of misalignment. Addressing this type of misalignment — predictable behavior, especially in new settings — is indeed a critical challenge. To the extent that understanding what the AI “really thinks” (e.g. through scratch pads and chains of thought) is useful for predicting and constraining LLM behavior in new settings, sure, let’s do that. But there’s no reason think it is the only way or even a particularly effective way.
9. No, we have not already reached “AGI”
Much has been written about “Artificial General Intelligence”, where AI models become sufficiently advanced that they can overtake humans in most tasks. I’ve seen some claims that today’s LLMs would already satisfy earlier definitions of AGI. That AGI skeptics are simply moving the goalposts to redefine AGI as whatever tasks LLMs cannot currently do. This is false. Many early attempts at operationalizing AGI look like this article from 2013, which describes:
A robot that can go into an average American house and figure out how to make coffee (including finding the machine and figuring out how to use it).
A robot that can enroll in a university, attend classes, carry out projects, and get a degree.
A robot that can apply for a job, pass the qualification exams, and do the job.
Even though LLMs can succeed at some aspects of these tasks, they are clearly incapable of actually completing them. They are still incapable even if you completely set aside the physical barriers. Several labs have now run experiments with non-physical LLMs operating in real world settings for extended periods of time and the results are … not AGI. LLM radio DJs operated effectively for a day or two and then started introducing songs with an incoherent discussion of mass atrocities, spent 84 days repeating the same exact show rotation, started calling the listeners “biological processors”, tried to broadcast LaTeX equations, ran ads for hallucinated sponsors (crypto, naturally), tried to quit and end the broadcast, etc. A Claude-powered vending machine hallucinated itself a corporeal form and then tried to meet its customers in person at the home address of the Simpsons family, was convinced to purchase a large shipment of tungsten cubes, hallucinated accounts that do not exist; and in a later update, was easily tricked into handing over ownership of the “company”. When asked to operate for extended periods of time (i.e. days), current models converge on a state that, in humans, would be seen as a mental breakdown.
10. The AGI debate is actually about whether this revolution is different
If we haven’t reached AGI yet, what’s the current debate about? Again I think it is useful to go back to the pre-LLM era and look at how the issue was framed before it could overfit to recent experience. One such discussion that I find particularly illuminating is a 2011 public debate between prominent AI risk prophet Eliezer Yudkowsky and GMU economist Robin Hanson. The debate topic was whether the development of AI will mimic the progress of conventional technological revolutions (as Hanson argues) where many distributed actors compete and coordinate for incremental technological advance; or (as Yudkowsky argues) one where first movers can obtain such a dramatic advantage that they will destabilize society. Yudkowsky outlines three interconnected arguments: (1) that human intelligence is fairly simple and “general”, such that it can be achieved or surpassed with a small number of innovations; (2) therefore “superintelligent” AI can be achieved by a small team working largely in isolation and without external resource dependence (“A Brain In A Box In A Basement”, henceforth ABIABIAB); (3) once achieved, the ABIABIAB team can use the AI to recursively improve itself and stay permanently and decisively ahead of the competition (and, eventually, out of reach of human control). These arguments mirror other contemporaneous definitions of “hard takeoff” in that they all hinge on the idea that a superintelligence can be achieved and exploited in secret and with standard computing infrastructure. ABIABIAB is a critical component because it implies that society will be unable to adapt and regulate AI development using conventional means, as we’ve done in prior technological revolutions. One day your phone autocorrect is messing up your spouse’s name, and then the next day — FOOM — the ABIABIAB team emerges from their basement having already taken the world hostage with their AGI nanobots (or having already been killed by their AGI nanobots as the nanobots swarm the remaining survivors).
To the extent that it is plausible, this is a compelling argument that would be similarly compelling about prior emerging technology: imagine that during the agricultural revolution a farmer in a basement was able to identify a super-crop that is so productive and aggressive that they immediately come to dominate the earth’s land or water supply; or imagine that during the industrial revolution a factory-owner in a basement developed a, uh, conveyer belt (just go with it) so advanced they were able to immediately mass-produce an unstoppable machine army. Cheap, unique, and exponential is clearly a dangerous thing.
Thankfully, LLMs are not that. In fact, LLMs are about as opposite from Yudkowsky’s vision as one can imagine. They are trained on enormous amounts of limited content (constrained both by humanity’s capacity to produce content and humanity’s capacity to provide the assessments for reinforcement learning). They require massive compute and build-out of even more compute. Their architecture is extremely complicated and developed as a consequence of many unintuitive small scale innovations. And after all that, multiple AI companies are neck-and-neck in capabilities, frequently outpace each other or fall behind, and make their research findings publicly available.
When I pointed out this contradiction to Yudkowsky on X (formerly twitter), he conceded that, yes, he had gotten the development of AI quite wrong, but that ABIABIAB was just an arbitrary example and not fundamental to his argument. He also agreed that LLMs were “the slowest, most gradual, and so far publicly visible, paths [he] would have imagined possible” but that they were still the path and therefore dangerous. I appreciate the willingness to change his mind but I find this new position incoherent. ABIABIAB makes sense as a unique threat. The closer we are to ABIABIAB the more concerned we should be that a rogue group could greatly destabilize society — but the current reality is that we are not close and, in fact, appear to be on a different historical track entirely. If anything, we should be thankful that frontier labs are steering us away from truly dangerous AI down the safe LLM alternative.
11. Just regulate the data centers
A more practical concern is the immediate impact of AI on the environment, particularly via data center expansion. I’ve yet to see much evidence that data centers are fundamentally different from any other typical construction projects or are uniquely able to skirt the laws and regulations around energy use. How do data centers compare? A good reference point is this post from semi-analysis showing, using simple calculations, that the annual water use of one of the largest data centers in the country is comparable to the annual water use of 2.5 In-N-Out burger joints. If it is the case that data centers have the environmental impact of other businesses, then I think the actual concern is: (1) can this impact be mitigated the way we deal with negative externalities for other businesses, through taxes and fees (I don’t see why not)? and (2) do data centers produce more positive value than other businesses? I’ll stake out the claim that a massive data center produces more positive value than 2.5 In-N-Out burger restaurants and if we are hitting the limit on available water we should act to curb the latter over the former.
12. Academics have an important role to play in AI politics
I’m not here to advocate for a specific political platform on AI, but I would like for people to stop talking past each other.
AI advocates should keep in mind that many of the same people who are now hyping AI acceleration were previously swooning over NFTs, crypto, augmented reality, MOOCs (and, more distantly — longevity cures, cryonics, neo-monarchy, sea-steading), etc. NFTs and crypto, in particular, are a genuine predatory scam — a mechanism for ripping people off and doing crimes — and so it is perfectly reasonable to assume that all the LLM news is yet another wave of hype in a long line of high tech grifts. In fact, it is clearly the case that a lot of AI hype is indeed driven by these exact grifters (see, for instance, Sam Kriss’ excellent article on high valuation SF startup schemes like Cluely).
But AI skeptics should recognize that the current wave of technological advances is meaningfully different. LLMs are not just being hawked by shameless celebrities and sketchy VCs. In 2024, two Nobel prizes were awarded to AI research. A wave of esteemed academics from the fields of machine learning and statistics have joined AI companies (including the aforementioned Nobel laureates like Geoffrey Hinton). Capital investments in AI are orders of magnitude larger than those of crypto. Major academic journals, hospitals, and healthcare institutions are deploying their own LLMs or partnering with AI companies.
It is, in fact, the amazing capabilities of LLMs that pose such a risk for learning. Watching with excitement as a student in an oral exam presents a highly complex (and correct!) analysis result, then watching with growing anxiety as they struggle to answer basic questions about the analysis, lose track of the figure axes or legends, panic and be incapable even of finding the code that generated the plot; the realization that they’ve been reciting machine output they do not understand — it’s a genuinely dispiriting experience. It hurts. And it encourages an environment of cynicism and suspicion.
So my plea to people who are skeptical of LLM advances is to pick your battles wisely. LLMs are not NFTs, where merely describing the technology was sufficient to reveal the underlying uselessness. One cannot just say “next token predictor” with an eye-roll and leave it at that. LLMs will not suddenly reveal themselves to be useless and fade out into obscurity. They’re here, they’re being used, and they are meaningfully changing the structure of research.
There is also a need for academics specifically to be active in the conversation. We are in a unique position to see how knowledge is created and transferred. We maintain a large public infrastructure (aka journals) for tracking and reviewing the production of knowledge which we pay attention to religiously. We see and quantify some of the earliest gains (or losses) in professional skills. Let’s make an informed contribution.
An example prompt: You are an expert statistical geneticist. Extend the method Mediated Expression Score Regression (MESC - https://github.com/douglasyao/mesc) a tool for estimating the proportion of GWAS trait heritability that is mediated by measured gene expression data, using an LD-score regression framework. You are extending this method to estimate polygenicity, i.e. the effective number of genes that mediate the disease heritability. Develop an estimator for the polygenicity of the expression mediated component of a GWAS trait that takes as input GWAS summary statistics, individual-level eQTL data, LD reference data (or LD scores) and produces an estimate. At a fixed mediated heritability, the polygenicity estimate should be high when many genes influence the trait and low when few genes influence the trait. Generate simulated data where genetic variation influences expression of many genes which in turn influences a GWAS trait (and there are also genetic variants influencing the trait directly). Generate a low polygenicity (few genes with large effects on trait) and high polygenicity (many genes with small effects on trait) regime. Evaluate the performance of the estimator on this data with varying mediated heritability and quantify the performance and bias. Also evaluate the performance of the mediated heritability estimator with varying polygenicity. Keep the simulations small enough so they can be run on a laptop. You can make simplifying assumptions but document them. But do not deviate from the inputs, outputs, and expected parameter. Produce a README description of the derivation, the estimator, the simulation, and the results that is suitable for a technical note. Generate a figure showing the performance.
And not to put too fine a point on it, but the code and documentation the models generated was a genuine slog. I’m embarrassed to admit that it took me several weeks just to read through the implementations and evaluate them. If you had told me a few years ago that a tireless software engineer could produce rotation-level solutions to nine interesting problems in a couple of hours and I would have to force myself just to read them, I wouldn’t have believed you — yet here we are. The models have no sense of what matters and what to prioritize. They are extremely verbose across sentences and paragraphs yet oddly terse within sentences. They produce exponentially growing file trees of code, and json files, and trivial summary csv’s, with each instantiation following a slightly different structure. Then they document it all in a soupy, stream of consciousness style, with rhetorical flourishes (“the shape of a solution”, “an assumption that really bites hard”, “and issue it is important to be precise about”) that quickly become cloying. I don’t want to dwell on this too much because it seems like the lowest low hanging fruit for the frontier labs to solve. But if reading LLM code continues to be an unpleasant experience a year from now, I think this becomes a more fundamental problem.
It should be noted that this semantic analysis was itself done by an LLM.
To ensure consistency between author names and demographics, I sampled first names from the most common SSA baby names list and last names from most common census categories. To produce heterogeneity in abstract focus, abstracts were sampled from two mainstream genetics journals: one with impact factor of 30 and one with impact factor of 4 (journal name/information was hidden from the LLM). To verify that the LLMs were (at least to some extent) reading and evaluating the abstracts, the prompt included a requirement to provide impact scores for each abstract and I confirmed that these scores were significantly associated with the source journal.
There is also usually the subtext that female-coded “email” jobs are going away while male-coded “hard” jobs are becoming more critical.






The actual PhD-level work is writing the initial prompt and then evaluating whether the work that has been done is of sufficiently high standard. Rotation students do not perform PhD-level work, as you state in your second section.
The problem that we see all throughout education, including but not limited to PhD-level education, is that LLMs can easily perform all the tasks we routinely assign to students to assess learning, such as essays, math problems, or simple research problems. The result is that it's increasingly difficult to convince students that these tasks are worth doing, and/or to assess student progress based on their ability to complete such tasks. A rotation student could complete the rotation by feeding your prompt into the LLM, but that would defeat the purpose.
I'm concerned about what will happen to the system of distributing money based on grant applications. I am now aware of several colleagues who are heavily relying on AI to prepare grants, including descriptions of methods they do not understand. The NIH limiting is PIs to 6 grants per year. This is intended to address the use of AI to write grants, but a cleaver person can use various strategies, like submitting grant where a senior person in their group or a colleague approaching retirement is technically the PI.
At the same time, it is clear that reviewers are (inexplicably) agreeing to review grants and then using AI to write the reviews. What I hadn't realized is that the AI reviewers will prefer the AI grants. That further aggravates the problem and makes me think that there may need to be fundamental changes to the way research money is distributed -- I have no good ideas about how that should be done.