Poor correlation of the Journal Impact Factor with scientific impact

Definition of the Journal Impact Factor
The Journal Impact Factor (JIF or short IF) of Journal X is the number of citations found from all journals & proceedings (in the Web of Science core collection) published in 2017 to articles in Journal X, divided by the number of articles that were published in the two previous years (2016 and 2016). For the JIF, only journals and proceedings count (not books) and within journals only original articles and reviews (editorials, letters and meeting abstracts do not count).

As one can easily see, there are several factors in this description that lend themselves to interpretation and manipulation by humans. First, any writing in a scientific journal needs to be classified (e.g. whether it is an article or a letter) and that classification is made by humans. Furthermore, the selection of what is included in the Web of Science core collection is not a fixed constant, but the list is updated dynamically by humans.

Web of Science versus Scopus
So what about journals that are not in the WoS core collection? I have published e.g. an article in a German-language journal (Lymphologie in Forschung und Praxis). This Journal is not included in any WoS collection. Despite initial reservations, this review article has meanwhile gathered 8 citations, which is a remarkable success given that the median citation count of a scientific publication is four (source; see also this blog post). How can I report the JIF when some grant application requires me to list it?

Luckily, there is still competition to the Impact Factor, namely the Cite Score. That number was invented by the World's largest scientific publisher Elsevier. Elsevier keeps also count of citations in the Scopus database. Scopus is less choosy when it comes to journal selection and most journals that are not listed in the WoS core collection are listed by Scopus.

ResearchGate Journal Impact?
So what do you do if your journal is not listed by either of the two big scientometric providers? There is still one other metric that you can use, namely ResearchGate. The RG Journal Impact is not well documented (at least I could not find much) but presumably uses a similar method. However, it is difficult to trust a metric if you do not know how it is calculated. I have published exactly one article that is not counted by WoS or Scopus. However, this is due to fact that the journal (Frontiers in Biotechnology and Bioengineering) is young. Its publisher has announced that it will receive an impact factor still this year and that the Impact Factor will be quite nice. For the time being, I am using the RG Journal Impact (3.02).

Large discrepancies in the journal evaluation
Even though the way the cite score is calculated is very similar to the JIF, the resulting numbers can be surprisingly different. At the bottom of this text is a comparison of the JIF, Cite Score and RG Journal Impact for three different journals from the year 2015, in which I have published. One notable difference is that the JIF takes into consideration the 2 previous years, whereas the Cite Score considers the previous 3 years. However, that does not explain the majority of the differences. The Cite Score is also less choosy when it comes to the content type: it counts - unlike the JIF - also editorials and letters (as a rule of thumb it counts everything).

A weak correlation
The Impact factor is misused. It was developed to evaluate journals (for librarians to decide whether to subscribe to a journal or not) and has then subsequently been used to evaluate individual articles and even individual researchers. The correlation between the Impact Factor and the actual scientific impact (e.g. measured in terms of citations) has been steadily declining over the last decades (see e.g. this research: https://arxiv.org/abs/1205.4328). From my personal experience, I can confirm that this is true as the correlation between the JIF and the actual number of citation among my own publications is around 0.32. This is pretty bad if your goal is to use the JIF as a proxy to predict scientific impact in terms of future citations.

UPDATE
The 2018 Impact Factor for Frontiers in Biotechnology and Bioengineering is 5.122 (https://www.frontiersin.org/journals/bioengineering-and-biotechnology#). Congratulations!