Beyond Metrics: A Framework for Scholarly Evaluation in LIS, Communication, History, and Philosophy

PDF
Published: Dec 31, 2025
DOI: https://doi.org/10.26265/jiim.v10i2.42638
Keywords:
Peer Review Criteria Scholarly Publishing Library and Information Science Communication History Philosophy
Archontia Michaillidou
Department of Archival, Library & Information Studies, University of West Attica
Foteini Efthymiou
Department of Archival, Library & Information Studies, University of West Attica
Konstantinos Kyprianos
Department of Archival, Library & Information Studies, University of West Attica
Dimitrios Kouis
Department of Archival, Library & Information Studies, University of West Attica
Abstract

Purpose - This research aims to identify, document, and comparatively analyze the reviewer guidelines of the 80 most prominent academic journals, as ranked by Google Scholar Metrics, in the fields of Library and Information Science, Communication, History and Philosophy. The goal is to propose a unified, interoperable, and adaptable conceptual evaluation model that acknowledges disciplinary specificities while preserving scholarly autonomy.


Design/methodology/approach - A mixed-methods research design was employed, combining quantitative and qualitative content analysis of 22 sources (16 unique websites, 5 publisher responses, and 1 interview). Structured thematic coding was applied to the material, followed by the creation of four identical text analysis forms, each including eleven pairs of qualitative and quantitative questions aligned with 11 key article evaluation criteria. Descriptive statistics (means, medians, mode values, and standard deviations) were used to rank criteria, while qualitative comparisons were organized into thematic tables with direct excerpts to capture disciplinary similarities and differences.


Findings - The study revealed substantial commonalities across fields, particularly the importance of data adequacy, coherence of conclusions, and adherence to ethical standards. Notable disciplinary differences were also identified, such as rhetorical emphasis in Philosophy and technical precision in Library and Information Science. The analysis informed the development of an interoperable conceptual evaluation model structured around shared foundations with adaptable elements tailored to each field.


Originality/value - This research contributes an innovative conceptual evaluation framework that combines epistemological inclusivity with cross-disciplinary applicability. By enhancing transparency and supporting reviewers in interdisciplinary contexts, the model offers a foundation for future expansion into additional scientific domains and provides practical guidance for harmonizing article evaluation practices.

Article Details
  • Section
  • Research Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright Notice

Authors who publish with JIIM agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution Non-Commercial License that allows others to share the work with:
  2. An acknowledgment of the work's authorship and initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (preferably in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Author Biographies
Archontia Michaillidou, Department of Archival, Library & Information Studies, University of West Attica

Archontia Michaillidou is a graduate of Classical Philology from the Democritus University of Thrace (2022) and is currently pursuing a Master’s degree in Information Management in Libraries, Archives, and Museums at the Department of Archival, Library and Information Studies, University of West Attica (2023–present). This marks her first involvement in academic research, with a growing interest in library and information science. Her academic focus includes the humanities, history, literature and philosophy.

Foteini Efthymiou, Department of Archival, Library & Information Studies, University of West Attica

Foteini Efthymiou, a Special Technical Laboratory Staff in the ALIS Department since 2019 and a PhD Candidate, supports lab work in cataloging and information literacy courses. She is a member of the Information Management Research Lab, participating in funded research. She worked as a librarian at the NTUA Central Library (1999–2019). She holds a bachelor’s degree in Librarianship (T.E.I. of Athens, 1994-1998) and a Master of Arts in Librarianship (The University of Sheffield, UK, 2004-2005). Her research interests include media and information literacy, educational models and technologies, cataloging, and scholarly communication, with multiple conference presentations and published works.

Konstantinos Kyprianos, Department of Archival, Library & Information Studies, University of West Attica

Konstantinos Kyprianos is an Associate Professor at the Department of Archival, Library & Information Studies of the University of West Attica. He holds a PhD in Library and Information Systems from the Ionian University and a degree in Librarianship from the TEI of Athens. He also has two postgraduate degrees, one in Informatics (University of Piraeus) and one in the Management of Cultural Units (Hellenic Open University). He has worked in various libraries of the private and public sector and participated in the NSRF 2007-2014 digital convergence program at the library of the University of Piraeus. His areas of scientific interest and publications are bibliographic description standards, semantic web, openly linked data, and digital libraries.

Dimitrios Kouis, Department of Archival, Library & Information Studies, University of West Attica

Dr Dimitrios Kouis received his Diploma in Computer Engineering and Informatics from the University of Patras and his PhD from the National Technical University of Athens. He was the technical coordinator for various projects for the Hellenic Academic Libraries Link, from 2004 up to 2016. Currently, he is an Associate Professor and the President at the Archival, Library and Information Studies Department of the University of West Attica of Athens. He is also a member of the Information Management Research Lab. He has been involved in several European and national R&D projects in the fields of information technology (RACE II, ACTS, IST, FP6, FP7, H2020).

References
Tennant, J. P., & Ross-Hellauer, T. (2020). The limitations to our understanding of peer review. Research integrity and peer review, 5(1), 6. https://doi.org/10.1186/s41073-020-00092-1
Hamilton, D. G., Fraser, H., Hoekstra, R., & Fidler, F. (2020). Meta-Research: Journal policies and editors’ opinions on peer review. Elife, 9, e62529. https://doi.org/10.7554/eLife.62529
Karhulahti, V. M., & Backe, H. J. (2021). Transparency of peer review: a semi-structured interview study with chief editors from social sciences and humanities. Research Integrity and Peer Review, 6(1), 13. eer Review (2021) 6:13 https://doi.org/10.1186/s41073-021-00116-4
Ross-Hellauer, T., & Horbach, S. P. (2024). Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review. Research Evaluation, 33, rvae004. https://doi.org/10.1093/reseval/rvae004
Zhang, Y., & Wang, Y. (2024). Understanding delays in publishing interdisciplinary research. Information Processing & Management, 61(5), 103826. https://doi.org/10.1016/j.ipm.2024.103826
Okamura, K. (2019). Interdisciplinarity revisited: evidence for research impact and dynamism. Palgrave Communications, 5(1). https://doi.org/10.1057/s41599-019-0352-4
Newman, J. (2024). Promoting interdisciplinary research collaboration: A systematic review, a critical literature review, and a pathway forward. Social Epistemology, 38(2), 135-151. https://doi.org/10.1080/02691728.2023.2172694
Kim, K., Kogler, D. F., & Maliphol, S. (2024). Identifying interdisciplinary emergence in the science of science: combination of network analysis and BERTopic. Humanities and Social Sciences Communications, 11(1), 1-15. https://doi.org/10.1057/s41599-024-03044-y
Halliday, L. (2001). Scholarly communication, scholarly publication and the status of emerging formats. Information research, 6(4), 6-4. https://informationr.net/ir/6-4/paper111.html
Stock, W. G., & Stock, M. (2013). Handbook of information science. Walter de Gruyter.
Eisenberg, M. B. (2008). Information literacy: Essential skills for the information age. DESIDOC journal of library & information technology, 28(2), 39-47. https://www.indianjournals.com/ijor.aspx?target=ijor:dbit&volume=28&issue=2&article=005&type=pdf
Floridi, L. (2015). The onlife manifesto: Being human in a hyperconnected era. Springer nature. https://library.oapen.org/bitstream/handle/20.500.12657/28025/1001971.pdf
Van Dijck, J., Poell, T., & De Waal, M. (2018). The platform society: Public values in a connective world. Oxford university press.
Radsch, C. (2014). World trends in freedom of expression and media development. World Trends in Freedom of Expression and Media Development. Ed. Courtney C. Radsch. UNESCO: Paris, France. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2424905
Ruben, B. D. (2017). Between communication and information. Routledge. https://doi.org/10.4324/9781351294720
Waisbord, S. (2019). Communication: A post-discipline. John Wiley & Sons.
Song, H., Eberl, J. M., & Eisele, O. (2020). Less fragmented than we thought? Toward clarification of a subdisciplinary linkage in communication science, 2010–2019. Journal of communication, 70(3), 310-334. https://doi.org/10.1093/joc/jqaa009
Breuer, J., & Haim, M. (2024). Are we replicating yet? Reproduction and replication in communication research. Media and Communication, 12. https://doi.org/10.17645/mac.8382
Ivory, J. D. (2024). Remembering reasons for reform: A more replicable and reproducible communication literature without the rancor. Media and Communication, 12. https://doi.org/10.17645/mac.7852
Hammarfelt, B., Hammar, I., & Francke, H. (2021). Ensuring quality and status: peer review practices in Kriterium, a portal for quality-marked monographs and edited volumes in Swedish SSH. Frontiers in Research Metrics and Analytics, 6, 740297. https://www.cogitatiopress.com/mediaandcommunication/article/view/7852
Müller, K., Salö, L., & Sörlin, S. (2024). Quality from within: Entry points to research quality in the humanities. Research Evaluation, 33, rvae029. https://doi.org/10.1093/reseval/rvae029
Heesen, R., & Bright, L. K. (2021). Is peer review a good idea?. The British Journal for the Philosophy of Science, 72(3), 635-663. https://doi.org/10.1093/bjps/axz029
Pickard AJ. (2013). Historical research. In: Research Methods in Information. Facet; 2013:167-178. https://doi.org/10.29085/9781783300235.019
Sytsma, J., & Livengood, J. (2015). The theory and practice of experimental philosophy. Broadview Press.
Cova, F., Strickland, B., Abatista, A., Allard, A., Andow, J., Attie, M., ... & Zhou, X. (2021). Estimating the reproducibility of experimental philosophy. Review of Philosophy and Psychology, 12(1), 9-44. https://doi.org/10.1007/s13164-018-0400-9
Andow, J. (2016). Qualitative tools and experimental philosophy. Philosophical Psychology, 29(8), 1128-1141. https://doi.org/10.1080/09515089.2016.1224826
Li, J., & Zhu, X. (2023). Twenty years of experimental philosophy research. Metaphilosophy, 54(1), 29-53. https://doi.org/10.1111/meta.12602
Spencer, R. (2014). Philosophical Approaches to Qualitative Research Renée Spencer, Julia M. Pryce, and Jill Walsh. The Oxford handbook of qualitative research, 81. [Full access]
Dhillon, P. (2021). How to be a good peer reviewer of scientific manuscripts. The FEBS journal, 288(9), 2750-2756. https://febs.onlinelibrary.wiley.com/doi/full/10.1111/febs.15705
Bonaccorsi, A. (2018). Peer review in social sciences and humanities. Addressing the interpretation of quality criteria. In The Evaluation of Research in Social Sciences and Humanities: Lessons from the Italian Experience (pp. 71-101). Cham: Springer International Publishing. https://link.springer.com/chapter/10.1007/978-3-319-68554-0_4
Haines, S. T., Baker, W. L., & DiDomenico, R. J. (2017). Improving peer review: What journals can do. American Journal of Health-System Pharmacy, 74(24), 2086-2089. https://academic.oup.com/ajhp/article-abstract/74/24/2086/5102718
Bornmann, L. (2011). Scientific peer review. Annual review of information science and technology, 45(1), 197-245. https://dl.acm.org/doi/abs/10.5555/2766865.2766877
Aggarwal, R., Louie, A. K., Morreale, M. K., Balon, R., Beresin, E. V., Coverdale, J., & Brenner, A. M. (2022). On the art and science of peer review. Academic Psychiatry, 46(2), 151–156. https://doi.org/10.1007/s40596-022-01608-1
Nicholas, K. A., & Gordon, W. S. (2011). A quick guide to writing a solid peer review. Eos, Transactions American Geophysical Union, 92(28), 233-234. https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2011EO280001
Davis, W. E., Giner-Sorolla, R., Lindsay, D. S., Lougheed, J. P., Makel, M. C., Meier, M. E., ... & Zelenski, J. M. (2018). Peer-review guidelines promoting replicability and transparency in psychological science. Advances in Methods and Practices in Psychological Science, 1(4), 556-573. https://journals.sagepub.com/doi/full/10.1177/2515245918806489
Brown, L. M., David, E. A., Karamlou, T., & Nason, K. S. (2017). Reviewing scientific manuscripts: A comprehensive guide for peer reviewers. The Journal of
thoracic and cardiovascular surgery, 153(6), 1609-1614. https://www.jtcvs.org/article/S0022-5223(17)30250-7/fulltext
Seeber, M. (2020). How do journals of different rank instruct peer reviewers? Reviewer guidelines in the field of management. Scientometrics, 122(3), 1387-1405. https://link.springer.com/article/10.1007/s11192-019-03343-1
Ali, P. A., & Watson, R. (2016). Peer review and the publication process. Nursing Open, 3(4), 193–202. https://doi.org/10.1002/nop2.51
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational researcher, 33(7), 14-26. https://journals.sagepub.com/doi/10.3102/0013189X033007014
Mayring, P. (2014). Qualitative content analysis: theoretical foundation, basic procedures and software solution. [Inproceedings]. Retrieved from https://www.ssoar.info/ssoar/bitstream/handle/document/39517/ssoar-2014-mayring-Qualitative_content_analysis_theoretical_foundation.pdf
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative health research, 15(9), 1277-1288. https://journals.sagepub.com/doi/10.1177/1049732305276687
Zandvanian, A., & Daryapoor, E. (2013). Mixed methods research: A new paradigm in educational research. Journal of Educational and Management Studies, 3(4), 525-531. J._20Educ._20Manage._20Stud.__2034525-531_202013_20-libre.pdf