Nexteam is sponsoring this newsletter. Please tell your friends and colleagues about this publication. Thank you.
A ChatGPT Alternative Is Now Open Source, And There Is A Catch
RLHF (Reinforcement Learning with Human Feedback) is a technique that can be used to improve language models such as Google's PaLM (which has 540 billion parameters) and ChatGPT. RLHF involves fine-tuning a language model with a dataset that includes prompts and then ranking the model's responses to those prompts. A "reward model" is then trained using the rankings to arrange the responses in accordance with preference. PaLM + RLHF is an open-source version of ChatGPT that combines PaLM with RLHF and can perform similar functions, but it requires significant amounts of data and resources to train and run. Several projects are underway to create open-source ChatGPT-like models using RLHF and other techniques, including those led by Scale AI, Hugging Face, EleutherAI, CarperAI, and LAION.
https://www.opensourceforu.com/2023/01/a-chatgpt-alternative-is-now-available-as-open-source/
What if C# were like Go? Open source bflat has ‘Go-inspired tooling’, now targets bare metal
Software engineer Michal Strehovský has released bflat 7.0.1, an open-source native compiler for C# that allows C# applications to run on bare metal and provides native ahead-of-time (AOT) compilation. Bflat targets Linux, Windows, and Android on x64 or arm64, or on x64 only, and can output either native code or .NET IL (Intermediate Language). It has two standard libraries, one that is compatible with .NET and another called Zero which does not have a garbage collector. Bflat is intended to give C# developers new options for creating small, dependency-free executables, particularly useful in the cloud and serverless environments. However, as an unofficial project, it is not recommended for use in production and does not currently support macOS.
Ruby backs WebAssembly for CRuby binaries
Ruby 3.2.0, a new version of the open-source dynamic programming language, includes initial support for WASI (WebAssembly System Interface), allowing developers to create CRuby binaries that can run in web browsers, serverless edge environments, and other WebAssembly/WASI embedders. The release also includes a production-ready YJIT (Yet Another Ruby JIT) compiler, which has been tested on production workloads for over a year and is 41% faster than the Ruby interpreter. Other updates in Ruby 3.2.0 include an improved Regexp (regular expression) matching algorithm, a syntax_suggest feature, and support for anonymous rest and keyword rest arguments.
https://www.infoworld.com/article/3684410/ruby-backs-webassembly-for-cruby-binaries.html
Serverless Production Readiness Checklist
This article includes several topics. Performance tests should be conducted on Serverless applications to identify and fix potential bottlenecks. In-memory caching or a DynamoDB cache can improve performance. Penetration tests should be run on the service, and an internal security review should be conducted with an AWS expert. The AWS Web Application Firewall should be used on API Gateways and CloudFront distributions, and Amazon Inspector should be enabled on AWS Lambda functions and layers. A vulnerability scanner should be used in the CI/CD pipeline, and IAM permissions and AWS best practices guidelines should be followed in the deployment framework. Canary deployments should be used for both AWS Lambda functions and dynamic configuration, and provisioned concurrency can be enabled for critical services to prevent cold starts. Continuous integration and delivery practices should be implemented to ensure efficient and reliable deployments.
https://www.ranthebuilder.cloud/post/serverless-application-productiozation-check-list
Tips for keeping your Lambda functions secure
It is important to make sure the environment is secure by following the principles of least privilege and by authenticating and authorizing application users and protecting API endpoints. Data should be encrypted and stored securely, and the overall security of the environment should be continuously monitored.
https://awstip.com/tips-for-keeping-your-lambda-functions-secure-25349dd5d9df
Multi-Region Event-Driven Applications with EventBridge Global Endpoints
Events are the blood in our systems and if for some reason those events stop flowing our systems will stop working. That is why if we are looking to build a multi-region application we need to have a strategy to handle our events in the case of a regional failure. For that EventBridge Global Endpoints can help.
Side-by-side comparison of serverless databases
It covers several DBMSs and there is a comparison table at the end.
https://fauna.com/blog/comparison-of-serverless-databases
The 7 Challenges of Big Data Analytics
https://www.chaossearch.io/the-7-challenges-of-big-data-analytics-with-thomas-hazel
A series focused on the seven deadly (ie. costly & complex) challenges of big data analytics!
How to Increase Deployment Observability and Simplify Deployment Pipelines
This article discusses the importance of having observability in deployments, or the ability to understand what changes were made, by whom, and why, in order to predict the effects of those changes on the system. The author discusses the need for this information to be provided to stakeholders at the time of the deployment and the importance of answering the 5 Ws (Who, What, When, Where, Why) in order to understand the root cause of any issues. The author then goes on to describe the challenges with using Azure DevOps and Slack integrations to achieve this level of observability and the solution of using APIs to get the desired information and send notifications to Slack.
The 12 Platform Challenges - recap
A collection of 12 blog posts that delve into various challenges faced by teams building an internal development platform. These posts cover a range of topics including budget and team size, cognitive load and user flexibility, and multi-cloud issues. There is plenty of material to keep you engaged and informed.
https://www.syntasso.io/post/the-12-platform-challenges-recap
Querying the GitHub archive with the ClickHouse playground
The ClickHouse Playground is a free hosted environment for exploring ClickHouse, which allows users to run queries on the GitHub events archive, which contains a copy of the GH Archive - a project that archives and makes available the public GitHub timeline. The archive dates back to February 2011 and is updated continually. The ClickHouse demo table is updated with the latest archived data every 10 minutes. Users can access the API via curl and receive data in TSV or JSON format. This tutorial will show you how to use ClicHouse to query GitHub data.
https://til.simonwillison.net/clickhouse/github-explorer
Terraform — The Awesome Value Type “any” and Loops
This is a thorough article discussing the use of HCL, a programming language used in Terraform, with a focus on flexible loops and the "any" type. The post also includes a practical example.
https://awstip.com/terraform-the-awesome-value-type-any-and-loops-a6be039b1ce5
OTT TV Services
The TV Services department of Sky had been struggling due to changes in personnel and leadership, leading to a decrease in quality and an increase in technical debt. The Delivery Lead was introduced to the team and identified the main problems, including a divided team with cliques and a lack of belief in the Scrum process. To improve the situation, the team implemented new policies and ways of working, added a Technical Lead and Senior Developer to the team, and adopted a shared responsibility for technical analysis among all team members. The team also introduced a Product Owner to ensure that user stories were well defined and met their Definition of Ready. These changes led to improved trust and communication within the team, and helped them successfully deliver product enhancements for the German market.
https://nexteam.co.uk/post/case-studies/ott-tv-services
conc
: better structured concurrency for go
Conc is a library that aims to improve upon Go's already strong concurrency capabilities by providing additional abstractions and techniques for handling goroutines and panics. It offers features such as concurrent mapping and iteration, panic catching, and pools to help prevent goroutine leaks and unhandled panics.
https://github.com/sourcegraph/conc
The Go libraries that never failed us: 22 libraries you need to know
A list of recommended libraries for various tasks in Go programming. The libraries are grouped into categories such as data structures, testing, and HTTP servers. The author provides a brief description of each library and its use cases.
https://threedots.tech/post/list-of-recommended-libraries/
Newsletter sponsor: Nexteam
Technology, Experience, Delivered.