Author :
Abstract
Veri üretimi her geçen gün katlanarak artmakta. Üretilen bu verinin hacmi, çeşitliliği ve hızı mevcut verilerden daha fazla olup analiz edilmesi ve yorumlanması oldukça zordur. Milyarlarca ağa bağlı sensörler akıllı telefonlar, otomobiller, sosyal medya siteleri, dizüstü bilgisayarlar, PC'ler ve endüstriyel makineler gibi verileri çalıştıran, üreten ve ileten cihazlara yerleştirilmiştir. Bu tür kaynaklardan elde edilen veriler yapılandırılmış, yarı yapılandırılmış veya yapılandırılmamış formattaki verilerdir. Geleneksel veritabanı sistemleri bu veri türlerini işlemede yetersiz kalmaktadır. Bu nedenle yeni teknolojilere ihtiyaç duyulmuştur. Bugün geliştirilen teknolojiler büyük veri setlerini; toplama, işleme, analiz etme ve görselleştirmede oldukça başarılıdır. Bu teknolojiler özellikle yapısal olmayan büyük veri setlerini kolayca analiz ederek, şirketlere büyük avantajlar sağlamaktadır. Bu çalışmanın amacı, Büyük Veri Analizinde kullanılan Hadoop ve Spark teknolojilerinin yapılarını tanıtmak ve bunların şirketler açısından sağladığı avantajları ele almaktır.
Keywords
Abstract
Data production is increasing day by day. The volume, diversity and speed of this generated data is more than the available data and is difficult to analyze and interpret. Billions of networked sensors are embedded in devices that run, generate and transmit data such as smartphones, automobiles, social media sites, laptop computers, PCs and industrial machines. The data obtained from such sources is the data in the structured, semistructured, or unstructured form. Traditional database systems are insufficient to process these data types. That's why new technologies are needed. The technologies developed today are large data sets; Collecting, processing, analyzing and visualizing. These technologies provide great advantages for companies, especially by easily analyzing large unstructural data sets. The purpose of this study is to introduce the practices of Hadoop and Spark technologies used in Big Data Analysis and to discuss the advantages they provide for companies.
Keywords
- Apache Hive, (2017). https://hive.apache.org/, (Erişim: 25.05.2017)
- Apache Hive, (2017). https://hive.apache.org/, (Erişim: 25.05.2017)
- Apache Kafka, (2017). https://www.tutorialspoint.com/apache_kafka/index, (Erişim: 15.03.2017)
- Apache Pig Tutorial, (2017). https://www.tutorialspoint.com/apache_pig/apache,(Erişim: 10.06.2017) Apache Spark Tutorial, (2017). www.tutorialspoint.com/apache_spark/, (Erişim: 10.05.2017)
- Apache Sqoop, (2017). http://sqoop.apache.org/, (Erişim: 10.04.2017)
- Brad, H., (2017). “Understanding Hadoop Cluster and the Network”, /01.01.2017/http://bradhedlund.com/2011/09/10/understanding-hadoop-clustersand-the-network/, (Erişim: 15.10.2017) Confluence, (2016). cwiki.apache.org/confluence/display/Hive/HCatalog/, (Erişim: 20.02.2017)
- Databricks, (2016). https://databricks.com/spark/about/20160102/, (Erişim: 10.05.2017)
- Datafloq, (2017). https://datafloq.com/read/in-brief-what-is-datamonetization/, (Erişim: 12.04.2017)
- Datafloq, (2017). https://datafloq.com/read/unlimited-computing-swarmsensors-algorithms-world/2138, (Erişim: 20.06.2017)
- Dezyre, (2017) https://s3.amazonaws.com/files.dezyre.com/images/Tut orials/HDFS, (Erişim: 18.01.2017)
- Dezyre, (2017). https://www.dezyre.com/article/hadoop-componentsand-architecture-big-data-and-hadoop- training/114/, (Erişim: 10.07.2017)
- Edureka, (2017). www.edureka.co/blog/bigdatatutorial?utm_source, (Erişim: 20.06.2017)
- Edureka, (2017). https://cdn.edureka.co/blog/wpcontent/uploads/2014/03 /, (Erişim: 12.07.2017) Edureka, (2017). www.edureka.co/blog/5-reasons-to-learn-hadoop, (Erişim: 19.07.2017)
- Edureka, (2017). https://www.edureka.co/blog/big-data-leads-economic growth, (Erişim: 19.07.2017)
- Facebook, (2017). “Facebook for Developers” https://developers.facebook.com/ /, (Erişim: 15.02.2017)
- Gates, A., (2011). “Data flow scripting with Hadoop: Programming Pig”, First Edition. 1005 Gravenstein Highway North, Sebastopol, CA 95472.: O’Reilly Media.
- Gürsakal N., (2014). ‘‘Büyük Veri’’ Genişletilmiş 2. Baskı, Dora, Bursa, ISBN:978-605-4798-803.
- Hook, L., (2016). “saudi wealth fund takes $3.5bn Uber stake”, JUNE 1, 2016, https://www.ft.com/content/a7e31c58-282c-11e6-8b18-91555f2f4fde, (Erişim: 20.10.2017)
- Hortonworks, (2017). https://hortonworks.com/apache/hive/#section_5, (Erişim: 10.06.2017)
- Kara, M., (2017). “Airbnb, 30 milyar dolar değerlemeden sonra karlığa ulaştı”, 27 Ocak 2017,https://webrazzi.com/2017/01/27/airbnb-30-milyar-dolar-degerlemeden-sonra-karliliga-ulasti/ (Erişim: 20.10.2017)
- Lisa, A., (2012). “Big Data Pushback? 86 Percent of Americans Vote No On Tailored Political Ads”.McKinsey Global Institute, (2011). “Big Data Report”, https://bigdatawg.nist.gov/pdf/MGI_big_data_full_report.pdf, (Erişim: 10.02.2016)
- Mohammad, H. F., (2011). ‘’Data Intensive Query Processing For Semantic Web Data Using Hadoop And MapReduce’’, Thesis, The University Of Texas At Dallas, s.7.
- Mukhammadov, R., (2013). ‘’A scalable database for a remote patient monitoring system’’, Master of ScienceThesis, School of Information and Communication Technology (ICT) KTH Royal Institute of Technology Stockholm, Sweden.
- Murthy, A., Markha, J., Vavilapalli, V. K., and Eadline, D., (2014). “Apache Hadoop YARN”.https://www.puurdata.nl/wpcontent/uploads/Apache.Hadoop.YARN _.Sample.pdf, (Erişim: 10.09.2017) Oğan, Ö., (2014). ‘’Büyük Veri Denizi’’, Elma yayınevi, Ankara, syf. 21.
- Ohlhorst F., (2013). “Big data analytics: turning big data into big Money”, New Jersey.
- Robert, R., (2006). “Representational State Transfer (REST)”, In Pro PHP XML and Web Services, 633–72. Springer.
- Slidershare, (2016). slidehshare.net, (Erişim: 12.01.2016)
- Thomas, D., (2014). “big data @ work, Türk Hava Yolları Yayınları”, Çeviren: Müge Çavdar, syf.127.
- Tutorialspoint, (2016). https://www.tutorialspoint.com/apache_spark/images/components_of_spark. (Erişim: 10.10.2016)
- Vaccari, C., (2014). “Big Data in Official Statistics”, University of Camerino, Doktora of Philosophy in İnformation Science and Complex System-XXVI Cycle, School of Science and Technology.
- Vavilapalli, V., (2013). “Apache hadoop yarn: Yet another resource negotiator”, in Proceedings of the 4th annual Symposium on Cloud Computing. ACM.
- WebHCat, (2016). https:/cwiki.apache.org/confluence/display/Hive/WebHCatUsingWebH, (Erişim:
