Reviews and feedback

The second review falls at the end of week ten. Each student has forty minutes: fifteen to present and twenty-five for discussion. The committee for each thesis sits at the front, with Leclerc, the other three students, a few younger students who have come to watch, and the readers from outside. Leclerc prepares the class the week before. 'A review is not an exam,' she says. 'It is the best free consulting you will ever get. Six people who know things you do not will spend twenty-five minutes thinking about your question. Your job is to make that time useful: show them what you need help with, listen, and write everything down.'

What a review is for

A thesis review has a different purpose from a studio review. In a studio review, critics judge a finished design. In a thesis review, especially early, the committee is judging the question and the research: whether the question is clear and worth asking, whether the methods are sound, whether the student knows the literature, and whether the explorations so far suggest the question can be answered by design. They will also judge whether the student can carry the work: whether the schedule is realistic and the scope manageable. A student who presents a beautiful rendering at the second review has usually misunderstood what the committee needs.

Leclerc asks each student to prepare three things. A one-sentence statement of the question, put up first and left up throughout. A short account of what has changed since the first review and why. And a list of two or three specific questions for the committee, the things the student is unsure about and would most like advice on. 'If you do not tell them what you need,' she says, 'they will tell you what they think, which is not the same thing.'

Ingrid's second review

Ingrid's committee is Abernathy, her adviser; Morningstar, her second reader; and Tomas Okonkwo, the critic who has sat on reviews since first year, as external member. Flett comes as her reader from outside. Ingrid puts up her question, her literature map, the district map with zones rather than homes, the survivability graph, the four explorations and their scores, and the warm room model. She asks the committee three things: whether the heat store in the pavilion is the right answer to the survivability gap; whether the network should include more buildings; and how to present the community's role without overstating it.

The discussion is lively. Abernathy says the pumped heat store is right, but asks her to state the outage length she is designing for and why: 72 hours is a common planning figure, but she should cite where it comes from and test what happens at 96. Okonkwo says the pavilion is too big for its job: the laundry, waiting room and repair café are good everyday uses, but she has drawn them at a scale that makes the pavilion a small community hub next to the existing hub, which confuses the district. Morningstar asks whether she has met the community organization since the survey went out and whether they have seen the first results. And Flett, who has been quiet, says something that changes the design: 'Older people will not walk far in the dark, even if it is only ten minutes. They will wait at home until somebody comes. You need people who go to them.'

Leclerc, closing the discussion, adds one more comment: the history section in Ingrid's draft proposal, about warming rooms in the early twentieth century, is interesting but twice as long as the argument needs.

Listening and recording

The hardest part of a review is listening. Students defend, explain, and argue, and the committee's time goes into the student's defence rather than the committee's advice. Leclerc teaches a simple discipline: during the discussion, answer factual questions briefly, ask clarifying questions when a comment is unclear, and otherwise write. A classmate takes notes as well, so that the student can look at the critics rather than the page. After the review, within a day, the student types up every comment, with who made it, and sorts them into themes.

Ingrid sorts her comments into a tracker with columns for who said it, which theme it concerns, what was said and whether it has been dealt with. Some are quick: she cuts the history section in half the next morning, and adds a paragraph to the proposal citing the sources for the 72-hour planning figure and showing the 96-hour case. Others need thought, design work or a meeting.

# Feedback tracker: comments from the first review, grouped and checked off
comments = [
    ("Leclerc", "question", "say who the warm network is for", "done"),
    ("Abernathy", "methods", "state the outage length you are designing for", "done"),
    ("Morningstar", "ethics", "meet the community organization before the survey", "open"),
    ("Okonkwo", "design", "the pavilion is too big for its job", "open"),
    ("Flett", "community", "older people will not walk far in the dark", "open"),
    ("Leclerc", "proposal", "cut the history section by half", "done"),
    ("Abernathy", "design", "test the church hall as well as the gym", "open"),
]
from collections import Counter
by = Counter(c[1] for c in comments)
print("comments by theme:", dict(by))
open_items = [c for c in comments if c[3] == "open"]
print(f"{len(comments) - len(open_items)} of {len(comments)} done")
for who, theme, text, _ in open_items:
    print(f"  open [{theme}] {text} ({who})")

comments by theme: {'question': 1, 'methods': 1, 'ethics': 1, 'design': 2, 'community': 1, 'proposal': 1}
3 of 7 done
  open [ethics] meet the community organization before the survey (Morningstar)
  open [design] the pavilion is too big for its job (Okonkwo)
  open [community] older people will not walk far in the dark (Flett)
  open [design] test the church hall as well as the gym (Abernathy)

Three of seven comments are done within a few days. The four that remain are the important ones, and they are of different kinds. Morningstar's is about ethics and relationships: Ingrid has not yet returned to the community organization with the first results, as she promised. Okonkwo's and Abernathy's second are about design. Flett's is about the question itself.

Kinds of feedback

Leclerc helps the students sort feedback by what it asks of them. Some comments are corrections: a fact is wrong, a calculation has an error, a source is missing. These should simply be fixed. Some are suggestions: try this, look at that precedent, test another case. These should be considered and either taken or declined with a reason. Some are challenges: the committee member doubts the claim, the method or the value of the question. These must be answered with evidence or argument, in the proposal and at the next review. And a few are reframings: comments that change what the thesis is about. These are rare and valuable, and they should be taken seriously even when they are uncomfortable.

Flett's comment is a reframing. Ingrid's network assumed that vulnerable residents would come to the warm rooms; Flett says that many will not, and that the network must also go to them. Ingrid spends a week on it, rereading her interviews with that comment in mind, and finds that it is supported: six of the twelve older residents she interviewed said they would stay home and 'wait it out'. Her survey had asked how far people could walk, not whether they would. Her question changes from 'which buildings could keep vulnerable residents warm' to 'which buildings, and which people, could keep vulnerable residents warm', and her program adds a small team of trained neighbours, organized from the hub, who check on residents who stay home, bring them warm meals and batteries for their devices, and bring them in if they are at risk.

Conflicting advice

Committees disagree, and a student who tries to satisfy everyone ends up with a thesis that satisfies no one. Okonkwo wants the pavilion smaller; Abernathy wants Ingrid to test the church hall as a warm room as well as the gym, which would add another building to the network, not take one away. Leclerc's advice is to go back to the question. Does a smaller pavilion still close the survivability gap and still earn everyday trust? Does testing the church hall serve the question, or only enlarge the project? Ingrid decides that both comments can be answered together: the pavilion shrinks to the laundry, the waiting room and the heat store, losing the repair café, which can move into the hub; and the church hall's simulation, done in a day with her calibrated model, shows that it stays habitable longer than the gym because it is smaller and heavier, which strengthens its role for older residents. She writes both decisions and their reasons in her design log and in the proposal.

The other reviews

Wren's committee challenges her flood numbers: the hydrologist who is her reader from outside points out that she used river levels at the nearest gauge, which is upstream of a tributary, so her flood heights are too low. She must correct them, which raises her plinth. Callum's committee asks for more specimens: five beams per batch cannot support a design value, and Szymańska and the forester both say he needs at least twenty, or must rely on published values and use his tests only to show that his joints reach them. Joon-ho's committee is split: Bérubé likes the two light slots, while an external member, a developer who has converted two office buildings, says that the cost of cutting the floors will make the project unaffordable without public money. Joon-ho decides that the cost question belongs in his thesis and adds a cost study to his methods.

Exercise 10: run and use a review

• Prepare your one-sentence question, an account of changes since the last review and two or three specific questions for the committee.
• During the discussion, listen and write; ask a classmate to take notes too.
• Within a day, type up every comment with who made it, and sort the comments into themes.
• Classify each comment as a correction, suggestion, challenge or reframing, and decide how to respond.
• Where advice conflicts, go back to the question, decide, and record your reasons.

Self-check

• Did you tell the committee what you needed help with?
• Have you kept every promise you made to the people who helped your research?

Going back to the community

Morningstar's comment is the one Ingrid acts on first, because it concerns a promise. Two weeks after the review she returns to the hub with the first results of the survey, printed large and in plain language, with the district's main languages side by side. About forty people come to the Thursday lunch at which she presents them. She shows what the survey found and what it did not: how many households have nowhere to go, how many rely on powered equipment, which buildings people named as places they go for help, and the fact that renters in the large apartment buildings are under-represented. She asks whether the results match what people know, and two residents of the large buildings offer to help reach their neighbours for a second round.

The meeting is also where she explains how Flett's comment has changed the thesis, and where she first describes the check-in team. The response is immediate: three people volunteer on the spot, and the community organization's director says that they already keep an informal list of elders to call in cold weather, which could become the team's starting point. Ingrid realizes that part of her network already exists, and that her design should strengthen it rather than invent it. She writes in her log that the most useful thing a review produced was a reason to go back.