Add Peer Review Report Mark with airSlate SignNow

Add esign Peer Review Report

hello and welcome to this second webinar on peer review today's topic is how to write a professional peer review report we will look at logical flaws in academic arguments structure of a peer review report giving constructive feedback the audience for your report and finally editing and proofreading now the first section is on flaws in academic arguments because when you write a peer review report you can't just say you don't like the paper or it doesn't make sense you really have to pinpoint what is wrong because you are trying to help the editor select papers for the journal so the aim of peer review is to select the papers and also to help the author improve it because in that way you're helping to improve the research archive now some journals do look at the journalism aspect that's whether the research will be news headlines the next day so that's a potential importance potential impact they also look at the technical quality there is a second type of journals that only look at the technical quality and they don't care about the potential impact so that's the study quality and reporting quality ethics and integrity study quality includes whether the research question is relevant and the work hasn't been published yet and study design has to be good this study has to be ethical the methods have to be appropriate and completely described so other people can repeat the work and the data analysis has to be sound and the conclusions valid for reporting quality reviewers and editors look at the logic as well as the standard of the English and the presentation and the work has to adhere to guidelines there's international guidelines for human and animal work so that's under equator network.org and you have to make sure the word because a good fit for the journal in topic as well as structure and format so that means you have to be familiar with the journal guidelines so definitely check the journal aims and scope as well as the particular instructions for authors now you also have to double-check you don't have conflicts of interest that might interfere with the objectivity of your review you may have to excuse yourself this includes being a current coworker with the authors or for example working for the company that makes a product that the authors are studying also check that the area of expertise is correct and you do have the time to complete the review by the deadline given by the editor now in the first seminar on Monday I spoke about the structure of a research paper so here's a recap it includes the introduction methods results and discussion at the beginning there's the title and abstract and at the end there's end matter for example references and ethical decorations so together the shape of the logical organization looks like a king or a queen chess piece on a chess board so that does show that logic is very important it begins broad at the beginning because you give general background and then the topic and specific problem and it starts narrowing down to the specific aim and then the actual studies methods results and key findings which lead to the discussion section which opens out again and becomes broad as the author discusses the meaning of the findings and the solution to the problem and what the study means in the real world so the author has to make sure all these sections are present and flow logically and as a reviewer you also have to look for these now the conclusion at the end or the solution to the problem is the main claim of the paper but throughout the paper in each of those parts there are smaller claims so each claim is basically a proposition or a statement or some kind of argument that you have support for it's got to be supported by reasoning and by evidence and you balance out whether evidence is for or against now most things do need references and citations if it's not common knowledge then the author needs to site so you can mention this in your review there are different types of evidence that authors can cite so you need to know the difference primary literature is original research secondary is reviews and databases and tertiary is where the knowledge has become stable and accepted by the public so it's now entered reference books encyclopedias and textbooks primary and secondary consists of academic work if it's in a journal or monograph or book there's also now preprints which are papers that have been uploaded to servers they haven't been peer reviewed and they haven't been submitted to a journal but the author is asking for public comment before publication there's also conference proceedings and conference abstracts of course in the everyday world there are general things like books newspapers magazines and newsletters now the ones shown here can be grouped or called grey literature grey because sometimes they are difficult to get to hold off or they are privately published or the peer review process is unknown so this includes posters society publications government reports and websites now authors and reviewers should always judge the source quality and accuracy of the evidence seeing that that is the foundation of people's reasoning and claims but do beware of fake or unethical journals don't just trust it because it says it's a scientific journal there's a checklist at the check submit org that you can use to make sure a journal is trustworthy so what makes a good argument in the research paper we send as a claim and the author will be arguing with a reason for the claim with support or evidence for the claim which could be data from the tests and experiments or other people's data by way of citations and references but you need more than just the claim reason and evidence because other people might think of exceptions or opposites so authors should really also think about counterclaims the opposite case this is also called a rebuttal and the strength of this rebuttal depends on its own reason and evidence so the author can argue though against the rebuttal and try to minimize the effect or negate the effect but if some parts of the opposite case or exception or the rebuttal are true enough and valid the author may have to modify the conclusion or modify the claim we call this a qualifier you might use words to lower the certainty or strength of the conclusion for example instead of saying something causes something you might have to say something may cause or something may contribute to so there's three ways to address the counterclaims and to be a good reviewer you can see whether the author has these and also if you can argue against a point that the author hasn't thought of this is part of your review as well so the counterclaim could be rejected completely because it also has weak evidence or reasoning so this is called a counter rebuttal or a counter counter claim there's acknowledgement and rejection where you acknowledge the validity of the rebuttal but you can argue that in this case it doesn't apply or there's concession and qualification where you may need to modify the strength of your claim or conclusion so except some part of the counter-argument and you use some hedging or weakening words like may or might also watch out for ABC a stands for assumptions so check that your own assumptions and the author's assumptions are sound including the relevance and focus of the Assumption beers for bias is that bias in the study itself or in the reporting or does the person have a financial or personal relationship that might be a conflict of interest c is for contradictions so there's one part of the argument then say the opposites later on or is the language not consistent the problem and solution might actually not use the same wording and you've answered a different question now I'm going to try an experiment if you type in the chat box then type your answer to what you think is wrong with these arguments because you need to pinpoint and be familiar with different types of flaw in arguments here's the first one the study of X at how tall means his colleagues his or her colleagues was unreliable because they carelessly failed to use an appropriate design there other studies are thus unreliable so if you see there's a problem please type in the chat box what you think the problem is now this is called an ad hominem fallacy there's different types of logical flaw they're called fallacies and sometimes they have Latin names actually it just means a personal attack so when you are writing a review you should criticize the research not the researcher eric says they carelessly failed to use an appropriate design the reviewer could easily have said the study did not use an appropriate design there's actually a second flaw as well the last sentence says there other studies are thus unreliable this is called a genetic fallacy genetic meaning the origin of something so it's an origin bias it's actually prejudice because the reviewer here thinks just because one study was flawed all the other studies of that author or the group of authors are flawed but that's not true now here's another example the data in the study of X are unreliable because the study was unreliable now what's the problem here now you'll see the words a similar study and unreliable so this is a circular argument or begging the question or a circular fallacy and sometimes authors will define something but then they use the words of the definition inside the definition in this example the study had a small sample so the effect was too small to detect this is called a straw man fallacy because the two parts are not really linked the author's changed the meaning a little bit so all because the study had a small sample then they've used another meaning of small and they say the effect was also small but that doesn't really link together here's another example the study of X was flawed in either the sample size or the interrater reliability inter-rater reliability is whether the different researchers who repeated the work whether their results agreed but this one is a false choice fallacy because the choices presented are irrelevant or the forcing the reader to choose but there's no reason why the study is only thought in one thing or the other the sample size or inter-rater reliability in this example X found that the chemical induced multiple mutations in bacteria but wide study showed mice were unaffected okay this is a false comparison fallacy because the two things being compared are again totally unrelated having mutations in bacteria doesn't mean that mice would be affected in what way so the comparison is wrong and the last example because x found that the chemical induced multiple mutations in bacteria this chemical would be a useful bathroom cleaner okay this is called a non sequitur because the points just don't relate to each other there's no reason why bacteria mutating means that the chemical is a good bathroom cleaner bathroom cleaners don't work by mutating bacteria so watch out for these types of logical flaws in the work that you are reading now in a research article these would be common problems related to flaws in the claim reason or evidence the ideas or the sections may not be logically organized or linked there could be facts mixed with opinion so the evidence is getting mixed up with claims and there's faulty reasoning the methods and statistical tests may be inappropriate or incomplete or unclear the results might be incomplete or unclear or just repeats the information in the display items the author could have confused statistical and real-world significance or because two groups differ by a large percentage like 20% in the test conditions it doesn't mean that this has a real-world meaning or the author confuses Association and cause this is a causation fallacy for example in children the IQ scores could increase with the size of the child as they grow so it could increase with shoe size but that doesn't mean shoe size increases people's IQ the discussion could suddenly introduce new results or just repeat the results section or the author hasn't discussed unexpected or negative results or any limitations remember these are the counter claims or rebuttals or the conclusions may not be supported by the data at all or overstretch and be too general or too certain all the cited studies may be missing or not up-to-date so this will affect the quality of the evidence