cruise control build automation

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.

linkedin/cruise-control

Folders and files, repository files navigation, cruise control for apache kafka.

Introduction

Cruise Control is a product that helps run Apache Kafka clusters at large scale. Due to the popularity of Apache Kafka, many companies have bigger and bigger Kafka clusters. At LinkedIn, we have ~7K+ Kafka brokers, which means broker deaths are an almost daily occurrence and balancing the workload of Kafka also becomes a big overhead.

Kafka Cruise Control is designed to address this operation scalability issue.

Kafka Cruise Control provides the following features out of the box:

Resource utilization tracking for brokers, topics, and partitions.

Query the current Kafka cluster state to see the online and offline partitions, in-sync and out-of-sync replicas, replicas under min.insync.replicas , online and offline logDirs, and distribution of replicas in the cluster.

Multi-goal rebalance proposal generation for:

• Rack-awareness
• Resource capacity violation checks (CPU, DISK, Network I/O)
• Per-broker replica count violation check
• Resource utilization balance (CPU, DISK, Network I/O)
• Leader traffic distribution
• Replica distribution for topics
• Global replica distribution
• Global leader replica distribution
• Custom goals that you wrote and plugged in

Anomaly detection, alerting, and self-healing for the Kafka cluster, including:

• Goal violation
• Broker failure detection
• Metric anomaly detection
• Disk failure detection (not available in kafka_0_11_and_1_0 branch)
• Slow broker detection (not available in kafka_0_11_and_1_0 branch)

Admin operations, including:

• Add brokers
• Remove brokers
• Demote brokers
• Rebalance the cluster
• Fix offline replicas (not available in kafka_0_11_and_1_0 branch)
• Perform preferred leader election (PLE)
• Fix offline replicas
• Adjust replication factor

Environment Requirements

• The main (previously migrate_to_kafka_2_5 ) branch of Cruise Control is compatible with Apache Kafka 2.5+ (i.e. Releases with 2.5.* ), 2.6 (i.e. Releases with 2.5.11+ ), 2.7 (i.e. Releases with 2.5.36+ ), 2.8 (i.e. Releases with 2.5.66+ ), 3.0 (i.e. Releases with 2.5.85+ ), and 3.1 (i.e. Releases with 2.5.85+ ).
• The migrate_to_kafka_2_4 branch of Cruise Control is compatible with Apache Kafka 2.4 (i.e. Releases with 2.4.* ).
• The kafka_2_0_to_2_3 branch (deprecated) of Cruise Control is compatible with Apache Kafka 2.0 , 2.1 , 2.2 , and 2.3 (i.e. Releases with 2.0.* ).
• The kafka_0_11_and_1_0 branch (deprecated) of Cruise Control is compatible with Apache Kafka 0.11.0.0 , 1.0 , and 1.1 (i.e. Releases with 0.1.* ).
• message.format.version 0.10.0 and above is needed.
• The kafka_2_0_to_2_3 and kafka_0_11_and_1_0 branches compile with Scala 2.11 .
• The branch migrate_to_kafka_2_4 compiles with Scala 2.12 .
• The branch migrate_to_kafka_2_5 compile with Scala 2.13 .
• This project requires Java 11.

Known Compatibility Issues

• Support for Apache Kafka 2.0 , 2.1 , 2.2 , and 2.3 requires KAFKA-8875 hotfix.

Quick Start

• git clone https://github.com/linkedin/cruise-control.git && cd cruise-control/
• Browse https://github.com/linkedin/cruise-control/releases to pick a release -- e.g. 0.1.10
• Get and extract the release: wget https://github.com/linkedin/cruise-control/archive/0.1.10.tar.gz && tar zxvf 0.1.10.tar.gz && cd cruise-control-0.1.10/
• Initialize the local repo: git init && git add . && git commit -m "Init local repo." && git tag -a 0.1.10 -m "Init local version."
• ./gradlew jar (Note: This project requires Java 11)
• Copy ./cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-A.B.C.jar (Where A.B.C is the version of the Cruise Control) to your Kafka server dependency jar folder. For Apache Kafka, the folder would be core/build/dependant-libs-SCALA_VERSION/ (for a Kafka source checkout) or libs/ (for a Kafka release download).
• Modify Kafka server configuration to set metric.reporters to com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter . For Apache Kafka, server properties are located at ./config/server.properties .
• If SSL is enabled, ensure that the relevant client configurations are properly set for all brokers in ./config/server.properties . Note that CruiseControlMetricsReporter takes all configurations for vanilla KafkaProducer with a prefix of cruise.control.metrics.reporter. -- e.g. cruise.control.metrics.reporter.ssl.truststore.password .
• If the default broker cleanup policy is compact , make sure that the topic to which Cruise Control metrics reporter should send messages is created with the delete cleanup policy -- the default metrics reporter topic is __CruiseControlMetrics .
• Start ZooKeeper and Kafka server ( See tutorial ).
• (Required) fill in bootstrap.servers and zookeeper.connect to the Kafka cluster to be monitored.
• Capacity file is a JSON file that provides the capacity of the brokers
• You can start Cruise Control server with the default file ( config/capacityJBOD.json ), but it may not reflect the actual capacity of the brokers
• See BrokerCapacityConfigurationFileResolver configurations for more information and examples
• (Optional) set metric.sampler.class to your implementation (the default sampler class is CruiseControlMetricsReporterSampler )
• (Optional) set sample.store.class to your implementation if you have one (the default SampleStore is KafkaSampleStore )
• (Note) To emit Cruise Control JMX metrics on a particular port (e.g. 56666 ), export JMX_PORT=56666 before running kafka-cruise-control-start.sh
• (Verify your setup) Visit http://localhost:9090/kafkacruisecontrol/state (or http://localhost:\[port\]/kafkacruisecontrol/state if you specified the port when starting Cruise Control).
• Cruise Control will need some time to read the raw Kafka metrics from the cluster.
• The metrics of a newly up broker may take a few minutes to get stable. Cruise Control will drop the inconsistent metrics (e.g when topic bytes-in is higher than broker bytes-in), so first few windows may not have enough valid partitions.

Cruise Control provides a REST API for users to interact with. See the wiki page for more details.

How Does It Work

Cruise Control relies on the recent load information of replicas to optimize the cluster.

Cruise Control periodically collects resource utilization samples at both broker- and partition-level to infer the traffic pattern of each partition. Based on the traffic characteristics and distribution of all the partitions, it derives the load impact of each partition over the brokers. Cruise Control then builds a workload model to simulate the workload of the Kafka cluster. The goal optimizer explores different ways to generate cluster workload optimization proposals based on the user-specified list of goals.

Cruise Control also monitors the liveness of all the brokers in the cluster. To avoid the loss of redundancy, Cruise Control automatically moves replicas from failed brokers to alive ones.

For more details about how Cruise Control achieves that, see these slides .

Configurations for Cruise Control

To read more about the configurations. Check the configurations wiki page .

Artifactory

Published at Jfrog Artifactory . See available releases .

Pluggable Components

More about pluggable components can be found in the pluggable components wiki page .

Metric Sampler

The metric sampler enables users to deploy Cruise Control to various environments and work with the existing metric systems.

Cruise Control provides a metrics reporter that can be configured in your Apache Kafka server. Metrics reporter generates performance metrics to a Kafka metrics topic that can be consumed by Cruise Control.

Sample Store

The Sample Store enables storage of collected metric samples and training samples in an external storage.

Metric sampling uses derived data from the raw metrics, and the accuracy of the derived data depends on the metadata of the cluster at that point. Hence, when we look at the old metrics, if we do not know the metadata at the point the metric was collected, the derived data would not be accurate. Sample Store helps solving this problem by storing the derived data directly to an external storage for later loading.

The default Sample Store implementation produces metric samples back to Kafka.

The goals in Cruise Control are pluggable with different priorities. The default goals in order of decreasing priority are:

• RackAwareGoal - Ensures that all replicas of each partition are assigned in a rack aware manner -- i.e. no more than one replica of each partition resides in the same rack.
• RackAwareDistributionGoal - A relaxed version of RackAwareGoal . Contrary to RackAwareGoal , as long as replicas of each partition can achieve a perfectly even distribution across the racks, this goal lets placement of multiple replicas of a partition into a single rack.
• MinTopicLeadersPerBrokerGoal - Ensures that each alive broker has at least a certain number of leader replica of each topic in a configured set of topics
• ReplicaCapacityGoal - Ensures that the maximum number of replicas per broker is under the specified maximum limit.
• DiskCapacityGoal - Ensures that Disk space usage of each broker is below a given threshold.
• NetworkInboundCapacityGoal - Ensures that inbound network utilization of each broker is below a given threshold.
• NetworkOutboundCapacityGoal - Ensures that outbound network utilization of each broker is below a given threshold.
• CpuCapacityGoal - Ensures that CPU utilization of each broker is below a given threshold.
• ReplicaDistributionGoal - Attempts to make all the brokers in a cluster have a similar number of replicas.
• PotentialNwOutGoal - Ensures that the potential network output (when all the replicas in the broker become leaders) on each of the broker do not exceed the broker’s network outbound bandwidth capacity.
• DiskUsageDistributionGoal - Attempts to keep the Disk space usage variance among brokers within a certain range relative to the average Disk utilization.
• NetworkInboundUsageDistributionGoal - Attempts to keep the inbound network utilization variance among brokers within a certain range relative to the average inbound network utilization.
• NetworkOutboundUsageDistributionGoal - Attempts to keep the outbound network utilization variance among brokers within a certain range relative to the average outbound network utilization.
• CpuUsageDistributionGoal - Attempts to keep the CPU usage variance among brokers within a certain range relative to the average CPU utilization.
• LeaderReplicaDistributionGoal - Attempts to make all the brokers in a cluster have a similar number of leader replicas.
• LeaderBytesInDistributionGoal - Attempts to equalize the leader bytes in rate on each host.
• TopicReplicaDistributionGoal - Attempts to maintain an even distribution of any topic's partitions across the entire cluster.
• PreferredLeaderElectionGoal - Simply move the leaders to the first replica of each partition.
• KafkaAssignerDiskUsageDistributionGoal - (Kafka-assigner mode) Attempts to distribute disk usage evenly among brokers based on swap.
• IntraBrokerDiskCapacityGoal - (Rebalance-disk mode, not available in kafka_0_11_and_1_0 branch) Ensures that Disk space usage of each disk is below a given threshold.
• IntraBrokerDiskUsageDistributionGoal - (Rebalance-disk mode, not available in kafka_0_11_and_1_0 branch) Attempts to keep the Disk space usage variance among disks within a certain range relative to the average broker Disk utilization.

Anomaly Notifier

The anomaly notifier allows users to be notified when an anomaly is detected. Anomalies include:

• Broker failure
• Metric anomaly
• Disk failure (not available in kafka_0_11_and_1_0 branch)
• Slow brokers (not available in kafka_0_11_and_1_0 branch)
• Topic replication factor anomaly (not available in kafka_0_11_and_1_0 branch)
• Topic partition size anomaly (not available in kafka_0_11_and_1_0 branch)
• Maintenance Events (not available in kafka_0_11_and_1_0 branch)

In addition to anomaly notifications, users can enable actions to be taken in response to an anomaly by turning self-healing on for the relevant anomaly detectors. Multiple anomaly detectors work in harmony using distinct mitigation mechanisms. Their actions broadly fall into the following categories:

• fix - fix the problem right away (e.g. start a rebalance, fix offline replicas)
• check - check the situation again after a configurable delay (e.g. adopt a grace period before fixing broker failures)
• ignore - ignore the anomaly (e.g. self-healing is disabled)

Contributors 113

• Python 1.6%

You are using an outdated browser. Please upgrade your browser to improve your experience.

MSBuild with Cruise Control .NET

You want to set up a customizable, fully automated build process. You want your build process to incorporate FxCop, NUnit, NCover, etc. but you do not have the license for Team System. No problem.  You can do all of this with free tools leveraging MSBuild and Cruise Control .NET (CCNet) as the basis of your solution. This is not new news.  In fact, CCNet is probably the best known tool today to do automated builds. However, there are lots of different ways to implement your build process. I'm going to do a series of posts that provide a start to finish implementation of one way to set up your build server. The posts will eventually include custom MSBuild tasks, custom CCNet add-ins, 32 bit / 64 bit considerations, and building MSI files with MSBuild.  In this post, I will just start with the primary structure and build process.

To kick off the MSBuild process, you'll use the standard "msbuild" task provided by CCNet pointing to the custom MSBuild project file that we'll construct "ccnetMSBuild.proj":

The basic outline of the msbuild script will be this:

• Version assemblies in the current solution (details to come in a later post)
• Do a Debug build (if compile fails, stop build)
• Run FxCop on the solution
• Run NUnit on the solution
• Run NCover on the solution
• If everything has run successfully, do a Release build

First, we utilize the standard msbuild task to do a Debug build.  Why do a Debug build before a Release build?  There are a couple of reasons.  First, Debug is the default mode for the developer on their local systems.  We encourage the developers to run FxCop locally before checking in - it's most convenient for them to be able to do this straight from Debug mode without having to first rebuild in Release mode.  Additionally, we typically will set our NUnit projects to build only in Debug mode but not in Release mode because we do not want to deploy NUnit projects out to a production environment.  Again, keeping it in Debug mode simplifies everything here.

This is the standard structure - in the interests of keeping this post a reasonable length, I will expand on the details on my next post...

Steve Michelotti

Hi, I'm Steve Michelotti. I'm a passionate technologist, create courses for Pluralsight, and speak at developer events on web and cloud technologies

• Dapper: Getting Started
• ng-bootstrap Playbook
• Building Mobile Apps with Ionic, Angular, and TypeScript
• Microsoft Azure Cognitive Services: Face API
• AngularUI Fundamentals
• Yeoman Fundamentals
• Building Mobile Apps With the Ionic Framework and AngularJS
• .NET Micro ORMs
• Knockout Fundamentals

14 min read

Cruise control to setpoint – a model for understanding pid loops.

By Phil Zito on Jul 7, 2022 5:00:00 AM

Topics: Building Automation System smartbuildings PID Loops service bas projects services setpoint PI Loops

Hey folks, Phil Zito here and welcome back. In this post, we are going to be diving into PID loops, or as I like to call them, PI loops because the reality is, we very rarely use derivative in building automation. So, by the end of this post, you will have an understanding of what PID loops are, what proportional, integral, and derivative do, why we use them, and you will have an easy way for setting them up.

It still amazes me to this day how many very experienced folks struggle with PID loops. How it seems to be this kind of black magic, mystical art that is tuning PID loops, when in reality, it's not that difficult. So, we're going to be looking at the cruise control model. There are many other models, and this is definitely not a scientific model, but there's all sorts of models and approaches to tuning PID loops, and I find them to be unnecessarily complicated. So, we are going to be looking at, what I view, as a very uncomplicated way of tuning PID loops.

Let’s dive in. For the sake of simplicity, I’m going to refer to these as PI loops. I don't believe in using derivative in about 98% of the situations you're in. It just does not make sense. It adds a complicating factor. What I hope you'll see at the end of this post, or by the middle of this post, is that the majority reason why people struggle with PI loops is because they don't know how to calculate proportional.

  So, what happens is, imagine you're in a car, and this car is driving down the road, and it comes to a hill. This hill is going to introduce resistance, it's going to introduce what is called error to your control. What's going to happen is, let's say your car is supposed to be maintaining 55 mph: that's its setpoint. As you start to go up this hill, what is going to happen is the car is going to slow down to maybe 45 mph, which means your engine needs to increase. Your engine is your output. Well, what happens is there is a proportional difference between your actual desired setpoint and your process variable: you're actually going 45 mph and you desire 55 mph.

So, there's a difference, we have what is called an error, and that error in our case is 10 miles per hour. Now there is a proportional response to this. As we increase our error, our output should increase proportionately. On direct acting loops, which means as error increases, the output increases, there should be a proportional response. This proportional response is our P value.

What happens is, as we increase our error, which is difference between setpoint and process variable, we should naturally have a proportional output response. Now, if we have a proportional output response, but we're still not getting to setpoint over a period of time, that's when we introduce an integral value. That's our I , and integral meaning time. So, over a period of time, we are going to add more to our output.

So, we have a proportional response to our error, and then as our error maintains over time, we have an integral response. So, our equation kind of looks like this:

Output = P x Error + (I x Error + Accumulated I)

That is our equation that we're primarily concerned with.

Now, for those of you who are still confused and are thinking this makes no sense, don't worry. It will make sense by the end of this post.

The biggest area people get wrong is this proportional response. So, what we want to do is, we want to figure out two variables: our throttling range and our sensor span. Fortunately, for us, our throttling range is almost always 100%. There are scenarios where it's not 100%, but I'm not going to cover them in this post. 100% is typically our throttling range.

Okay, what does that mean? That means the range of the output, it's full throttled range. So, if I go from stop, to 100%, or fully go, that in most cases, it's going to be 0 to 100%. That'd be 0 to 100% valve command, 0 to 100% VFD command, or in our case of our motor, 0 to 100% RPM.

Now, we have our sensor span, and this is the controllable range of the sensor in which we are having a response. So, what I see a lot of people do is they will try to set up a PID loop for maybe DAT (discharge air temp), and we know that DAT maybe has a 10-degree range. So, our setpoint may be 55, and we know that we drift between 45 to 55.

  Now I'm just using simple numbers here, because in actuality, 55 would actually be the middle percentage, and we would do like 60 to 50, but that's more advanced. So, we have a 10-degree sensor span.

Maybe you're in a room, you're in a space, and you have 72 degrees and our max cooling of 76. So, we have a sensor span of four. Now what we do is we take the throttle range of 100% and we divide it by our sensor span, which in the case of our room is 4. In the case of DAT, 10 divided into 100, so that would be 10. So, 4 would equal 25, and 10 would equal 10. So, our P for room control would be 25.

That means for every degree of error, we would get 25% output. So in theory, if we were at 76 degrees and our setpoint was 72, we would have 100% damper command on our VAV box. So, in theory, if we're at 65 degrees DAT, and we're trying to maintain 55, we would have 100%. But if we're at 50 degrees DAT we would have a 50% output.

So, this is how you get your P value, and your P value, if you get anything right in loop tuning, it's getting that initial P value. If you get that initial P value right, you're going to be so far ahead of pretty much everyone else.

Now we have this I value. We have to understand how I accumulates. Inside a building automation controller, we have a processor, we have a CPU, and that CPU processes all of our logic. Typically, in building automation, the standard is to process logic once a second. So, our loop processes once a second. So, our I value processes, once a second because it is going to process every time the building automation controller runs through the loop.

So, in our scenario, we have a P value of 25, because we're doing room control. We have an error of 2. So, our initial output on cycle one is going to be 50%. On cycle 2, which is cycle 2, we have our I value that we introduce.

This is why I always recommend starting with a small I , so if we do a 0.1, that will give us an I value of .2. So, .2 is going to be our initial I value, and that's going to be our I calculated.

  Then we have our I accumulated. In cycle three, we're running it again. We're three seconds in and we still get that error. So, we're still 50% across the board in response to our error. We add another .2, but we also have our I accumulated from before. So now our resulting loop is 50.4.

If we go to cycle four, we now have an accumulated of .4, we have an I calculated of .2, and we get 50.6. This is how PID loops work. Until we actually start to get a negative error, we are not going to have this accumulated I go away.

So fast forward to maybe cycle 10. We’re at accumulated 2. But let's say our error now is 0. So, we have a 0 proportional output, we have 0 error, we have 0 calculated I , but we still have 2%. That's why you can be in a 0-error scenario and still have an output on your PID Loop.

However, when we move into a negative scenario now, so we have a -2 error, that is where we actually will see our I start to reduce. So, we'll have our accumulated I and -2 error x .01. At cycle 11, we would actually get a -.2, so we would have 0.2% negative as our output.

Now, certain loop manufacturers keep this from happening. They keep from what's called integral wind up and integral wind down . In older controllers, this was a big issue in that you would have an I accumulate, and it would wind past 100% load. You would have a spun-up loop, and this was notorious for happening when a piece of equipment was off, but the PID Loop was still trying to control it and it never could. You would have I accumulated, or an ID cumulated decreased, basically a negative or positive accumulated I . Then you would have to reset the PID loop.

That has largely been dealt with, with most manufacturers. They put in thresholds so that you can't go above 100% accumulated I can't go below 100% accumulated I . This is why you actually start with a small I . So what you'll learn over time, if you get proportional right, then how do you calculate I ?

Well, how I like to calculate I is I look at the speed of change. So in a space, we typically have 4 air changes in a space. That's by the way, where our 15-minute trend interval tends to come from. Some folks will say our 15-minute trend interval comes from data capacity, and that is true, but how did they find that data capacity? If you divide 4 into 60, you get 15.

Since trending zone temp was the most common trend in building automation, like you may have one discharge air temp, but you have 100 zone temps, we would trend them based on air changes. Since 4 air changes is the most common air changes for space per hour, then we came up with this 15-minute trend interval.

Well, 15 minutes, by the way, as well is an air change. It's when we reasonably expect our space to be at setpoint from a complete error scenario. So, we go from like 76 to 72, we reasonably expect that we can achieve that within 15 minutes. So, we have to say, we want our I to accumulate over that time period.

So, what I've done, and this is total swag, Phil Zito experience here. There's no real science behind this, but what I've done in these scenarios is I look at whatever my control is. So, in the case of a space, it’s typically 15 minutes, and in the case of discharge air pressure on an air handler, that's typically 60 seconds. I look at that, and I identify that my I , I need it to go across this range in that amount of time, and that's how I come up with kind of an I value.

So, if I know that an average loop runs every second, so 60 seconds, that gives us 60 accumulations. If I did .01 x 60, that gives me 6. So, every minute, I would get six accumulated I . So, if I get 6 accumulated I , and this is assuming a pretty basic error of 1. But if I get a 6 accumulated I over that, if I times that by 15, that gives me right about 90. So, pretty close to 100% I that I can get over a 15-minute period.

Now, if I go to 60 seconds, that's not going to get me there. So, that's why you typically will see like .1, which .1 x 60 would put us at 60. So, like .15, would be a starting integral point for a discharge air pressure loop. Over 60 seconds, we would actually get 90 accumulated I , assuming a basic error of 1. Right?

So, that's kind of how in my career I've come up with my range of I for my building automation PID loops.

Now I want to talk about a direct-acting versus a reverse-acting loop. A lot of people get this really complicated, but it's actually very simple. Direct-acting means as error increases, output increases. Reverse-acting means is error decreases, output increases.

So, what would scenarios be where we have a direct-acting versus reverse-acting loop?

A direct-acting loop scenario would be like zone temp, discharge air temp, etc. Reverse-acting loop scenario would be something like, as pressure decreases, because if pressure increases, we actually want to lower our output. So, as we get a lower error, as we decrease in the case of discharge air pressure, or hot water, where we drop below setpoint, those would be scenarios where we want to increase our output, reverse-acting loop.

Cooling, space temp, etc, in the case of cooling, these would be scenarios where we want to increase our output as we get too high.

Now the next thing I'm going to say about this is what is called a stop or startup value. Not every PID Loop manufacturer supports this, but where you would do this is when you want to preload something. So oftentimes in space control, we'll do a preload of 50%, because we have minimum ventilation requirements for spaces when they're occupied. So, we will preload our PID with 50, and then from 50% to 100%, we’ll actually use like a scalar object, and this will throttle us between max and min cooling. Then 50% to 0%, this is pretty much just min cooling right here, and then 50%, we adjust working with the balancer, etc to get whatever that is.

This isn't a hard and fast rule. Some folks don't do this for their PID loops. Some folks will actually have a PID loop that’s 0 to 100, and then it'll go into two scalar objects, one for a heating mode and one for cooling mode. Others will have a temperature driven heating and cooling mode that goes into a switch that then switches CFM setpoint.

There's a variety of different ways to approach this in your setting up of loops. Just realize that startup value, that preload value, is a potential approach to things. I don't like to use it; I personally like to have a single purpose loop. So, if I'm going to use a loop for cooling, I have a loop for cooling. If I'm going to have a loop for heating, I have a loop for heating, and I tune them accordingly, because the rate of transfer, as well as my sensor span in a heating scenario is different than my cooling scenario.

So, I like to have two different loops, and then I like to use a numeration, like switching between cooling or heating or ventilation mode, and then controlling my loops accordingly. That's how I prefer to program. That is my approach. That's not to say it's the only or the right approach. It is my approach.

So, there you have it. This should give you kind of a baseline understanding of PID loops, or as I like to call them, PI loops. I hope you feel more comfortable after reading this. I hope this feels less mystifying. I feel like people try to validate themselves by making this seem way more complicated than it actually is, and my hope is that after reading this, you feel very comfortable.

If you still are looking for support beyond this, I would encourage you to check out our PID Loop Tuning Course . As always, make sure comments and ask questions.

Thanks a ton, and take care.

Written by Phil Zito

Previous post.

Making Sense of Electrical Concepts Without Becoming an Electrician

How to Secure Your BAS - Part 1

Want to be a guest on the podcast.

Quick Links

• Course Catalog
• Corporate Training
• Prospective Students
• Get in Touch
• SBA Podcast
• BAS Bootcamp
• See if Your Employer Will Cover Training Costs

• Terms of Use
• Privacy Policy

Copyright © 2020 Smart Buildings Academy

Videos and Webinars

• Contact sales
• Trial software
• Description
• Full Transcript
• Related Resources

How to Simulate Automated Driving Systems: Adaptive Cruise Control

Published: 18 Aug 2020

Hello, everyone. In this video, we will discuss the core tasks in designing automated driving systems, and we will see how these operations can be integrated for developing adaptive cruise control feature. This is a reference example of Highway Lane Following feature from the Automated Driving Toolbox. We use MATLAB to write the core algorithms and Simulink to integrate and simulate these algorithms as a model.

Developing automated driving systems typically involves creating algorithms for perception, planning and control. These algorithms are then integrated and iteratively tested in a virtual environment with synthetic scenarios for fast, safe and cost-effective development. A common simulation workflow starts with the perception of surrounding environment using sensor models.

The automated driving toolbox contains blocks for configuring parameters and acquiring data from camera, radar and LIDAR sensors. The acquired sensor data is processed using available algorithms for detecting objects, including lanes, pedestrians, vehicles and more.

In this example, the Vision Detect variant block detects lanes and vehicles from camera sensor data. These detections from multiple sensors are then combined using sensor fusion algorithms. The Forward Vehicle Sensor Fusion block combines object detections from camera and radar sensors, manages the tracks of moving objects, and eliminates false detections.

Based on the object detections, the planning algorithms calculate and alter the ego vehicle's trajectory. The vehicle control system finally enables autonomous vehicles to follow the plan trajectory by specifying literal and longitudinal decision logic. This subsystem incorporates Model Predictive Control. The controller provides optimal control actions by using an internal vehicle model to predict future behavior, based on length and vehicle detections.

Now we need a scenario to simulate the algorithms. You can either choose one of the pre-built scenes or create one yourself. The Driving Scenario Designer app can be used to create, edit, and visualize scenes which include roads, vehicles and sensors. This is a pre-built scene, and its open look simulation shows the lead vehicle slowing down and resulting into a collision.

The Simulation 3D Scenario subsystem reads the previewed scene into Simulink and configures the model for a 3D simulation.

Next, to simulate the effects of control algorithms on a vehicle, we need the dynamic equations of the vehicle. The Vehicle Dynamics block specifies the dynamics model for the eco vehicle and outputs that vehicle's post and velocity based on the controller input. The Metrics Assessment block is used to assess system level behavior.

Now let's test the integration of these algorithms in the 3D simulation environment. You can visualize multiple views with overlays during the simulation. The lane and vehicle detections are displayed on top of image frames while eco vehicle velocity and relative distance metrics can also be plotted in real time. The sensor data and metrics are logged for plotting and analysis post-simulation.

The birds-eye scope and video show that the eco vehicle successfully detects lines and vehicles. Additionally, the controller processes the detections to stay in the lane, as well as avoid accidents by slowing down when needed. Thus, existing algorithms can be readily integrated and simulated in 3D environment for developing datas and automated driving features.

You can also simulate other pre-built scenarios with this example. Here is another example which demonstrates how to automate testing the highway lane following feature once integration is complete. You can try these and other reference examples, as well as build complex algorithms on top of the examples, by installing the Automated Driving toolbox. For more details, check out the links for the Automated Driving toolbox product page and reference examples.

Related Products

Automated driving toolbox.

Bridging Wireless Communications Design and Testing with MATLAB

Featured Product

• Request Trial
• Get Pricing

Related Videos:

View more related videos

Select a Web Site

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

You can also select a web site from the following list

How to Get Best Site Performance

Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.

• América Latina (Español)
• Canada (English)
• United States (English)
• Belgium (English)
• Denmark (English)
• Deutschland (Deutsch)
• España (Español)
• Finland (English)
• France (Français)
• Ireland (English)
• Italia (Italiano)
• Luxembourg (English)
• Netherlands (English)
• Norway (English)
• Österreich (Deutsch)
• Portugal (English)
• Sweden (English)
• United Kingdom (English)

Asia Pacific

• Australia (English)
• India (English)
• New Zealand (English)
• 简体中文 Chinese
• 日本 Japanese (日本語)
• 한국 Korean (한국어)

Contact your local office

Portfolio: CruiseControl

• Technologies

Java-based framework for software build automation, offering continuous software integration using the Java Virtual Machine. It was released in 2001 under a BSD-style license as a cross-platform tool. Integration with Apache Ant is supported. A daemon background process checks the revision control system on a regular basis, detects any changes to the code and, on identification of a change, generates an assembly and creates a status notification. Reports are managed by Jakarta Server Pages (JSP) and are displayed on a proprietary dashboard. A .NET version, CruiseControl.NET, also exists.

Comprehensive automated system test cycle

A utility package to provide maximum quality code and fail-safe operation of critical components. Module tests have been written. Test environment set up on a testing rig, with smoke testing, daily build and nightly test runs. The GUI is diagnosed by simulating user actions. Resource leaks are eliminated.

May 16, 2008

Bugzilla C# C++ CMS CruiseControl Django HTML Linux MySQL PHP Python SQL Squish SVN

• Testimonials
• Custom R&D
• Outsourcing
• Documentation
• Requirements
• Development

• skip navigation Telerik Test Studio Product Bundles DevCraft All Telerik .NET tools and Kendo UI JavaScript components in one package. Now enhanced with: NEW : Design Kits for Figma
• Search in Documentation
• Search in API Reference
• API Reference

Advanced Topics / Build Server

New to Telerik Test Studio? Download free 30-day trial

Telerik Testing Framework and CruiseControl.NET

1. what is cruisecontrol.net.

CruiseControl.NET is an automated integration server. The Server automates the integration process by monitoring the team's source control repository directly. Every time a developer commits a new set of modifications, the server will automatically launch an integration build to validate the changes. When the build is complete, the server notifies the developer whether the changes that they committed integrated successfully or not. For more information about what CruiseControl.NET is and isn't and how to obtain it refer to their website at cruisecontrolnet.org . The installer is available at sourceforge.net/projects/ccnet/files . At the time this document was written, version 1.8.0.0 was the most current version of CruiseControl.NET available. It's pretty easy getting your test code to build using CruiseControl.NET but there are a couple of tricks to getting it to run your Telerik Testing Framework tests as unit tests. Don't fear however, the way to get your tests to run and viewable in the Web Dashboard Application is all done by adding or changing a few things in two configuration files.

2. Installing CruiseControl.NET

Installing CruiseControl.NET is very straightforward. The only thing that needs to be different from the default is to uncheck "Install CC.Net server as Windows service" as shown in the screen shot below. Because Test Studio tests utilize UI automation, a lot of these tests will not work when you try to run them under a Windows service. Windows does not grant services permission to use the mouse or keyboard. As a result any test that uses the mouse or keyboard will fail... and most UI automation tests rely on one or both at some point during the testing procedure.

3. Create Required Directories

You will need to create a directory to perform the build in before doing your first build:

• Create a unique directory for your project builds to run in. Don't try to mix different builds in the same folder. There's a potential that different builds can interfere with each other (especially if filenames are the same between the two projects). Make the directory name unique enough to identify it from all other project builds such as 'C:\Project builds\Sydney v1.1'. Later you'll enter this folder path into the ccnet.config file.

4. Modifying the Server's ccnet.config File

This paper was written referencing CruiseControl.NET version 1.8.0.0. Hopefully whatever version you're running is compatible with what I'm about to describe. Also to understand the changes I'm about to describe you need to have a basic understanding of where the ccnet.config file is and what belongs in it. This configuration file controls the builds for all of the projects CruiseControl.NET projects. First we need to get the server building our test code.

4.1 Adding a Build Task to Your Project

With CruiseControl.NET you have two different build configurations you can use. It doesn't really matter as far as building and running Test Studio or Telerik Testing Framework tests. Choose whichever method you feel most comfortable with (but don't add both to ccnet.config because CruiseControl.NET will blindly execute both build tasks).

4.1.1 Building Using the DevEnv Task in Your ccnet.config File

To build using the Visual Studio task , you simply add the task configuration to your project file. The following example assumes you're using Visual Studio 2010. Refer to the CruiseControl.NET documentation for other settings to make it work with other versions of Visual Studio. It should look something like this:

4.1.2 Building Using the MSBuild Task

This method is pretty much the same but instead of a Visual Studio task we replace it with a MSBuild task . It should look something like this:

4.2 Adding a Run Tests Task

The program that runs Visual Studio unit tests is Microsoft's MSTest utility . This command line utility comes with Visual Studio. It has many command line options for specifying what project to run, what tests in that project to run, and where to log the results to. If you're using NUnit or something similar you can skip this section but be sure to add an appropriate NUnit (or whatever testing framework you use) test task to your project. An MSTest task is an "Exec" task to CruiseControl.NET. It should look something like this:

If you want to run a suite of tests, add them to a Visual Studio test list and use a buildArgs like this:

4.3 Publishing Test Results

You need to merge the generated .trx file from MSTest into the build results. Add something like this to the 'publishers' section of your project configuration:

4.4 Publishing Test Results

One caveat to using MSTest is that you must delete the .trx file it creates before doing another test run. Otherwise MSTest will fail indicating it was unable to create the output file. This is done via another "Exec" task in ccnet.config. Place this section just in before your Exec task that runs MSTest. It probably would be best to place it at the beginning of the build because if a build fails (which causes CruiseControl.NET to skip any remaining tasks) to compile, CruiseControl.NET will still pick up and merge any left over .trx file into the build log:

4.5 Publishing Test Results

Now you're ready to run your CruiseControl.NET server and let it start building and running tests. CruiseControl.NET can be run in two modes:

Command line mode

Windows service mode

There is a problem with using Windows service mode. Test Studio tests and Telerik Testing Framework tests require the browser be able to run and interact with the desktop. When running builds and tests under a Windows service, desktop interaction is disabled. As a result virtually no Test Studio or Telerik Testing Framework test can successfully run in this environment. It is best to log onto your server using an account that has access to your source control and VisualStudio permissions and start the CruiseControl.NET server in command line mode. Leave this user logged on all the time. If you logoff this user the CruiseControl.NET server will shut down and you won't get any builds.

In order for Test Studio and Telerik Testing Framework tests that interact directly with the desktop to work you'll need to leave your CruiseControl.NET machine logged on and displaying the desktop all the time. This means you'll have to disable the screensaver and never lock the computer. Yes, this presents a security risk to your company. If you can lock this computer up into a secure room (such as a datacenter) you will minimize this security risk.

5. Modifying the Web Dashboard's dashboard.config File

As configured out of the box, CruiseControl.NET's Web Dashboard is preconfigured to only display build header information and modification details. We have to modify the dashboard.config file to get it to display the results of our tests. This file is located in the folder C:\Program Files (x86)\CruiseControl.NET\webdashboard

5.1 Modifying the xslFileNames Section

This section is a list of .xsl files that will be used after every build to generate the build's Build Report that's displayed in Web Dashboard. CruiseControl.NET runs them in sequence as listed in this section.

If you are running Test Studio tests you will want to install and use the TestStudioMSTest2010Report.xsl file. Download the file from the previous link and place it in the folder C:\Program Files (x86)\CruiseControl.NET\webdashboard\xsl. Next we need to add TestStudioMSTest2010Report.xsl so that it looks something like this:

If instead you are running Telerik Testing Framework coded unit tests we need to add MsTestSummary2010.xsl and MsTestReport2010 so that it looks something like this:

5.2 Modifying the xslReportBuildPlugin Section

This section is used to generate a menu of the different detail reports you can display. After you make these changes you'll get something that looks like this:

Note the last entry in the list. To obtain this entry we need to modify this section in the dashboard.config file to look something like this:

After making the above changes, the next build will successfully display test results.

• Getting Started
• Virtual Trainings
• Feedback Portal

Copyright © 2024 Progress Software Corporation and/or its subsidiaries or affiliates. All Rights Reserved.

Progress, Telerik, and certain product names used herein are trademarks or registered trademarks of Progress Software Corporation and/or one of its subsidiaries or affiliates in the U.S. and/or other countries. See Trademarks for appropriate markings.

• Terms of Use
• Privacy Center
• Security Center
• License Agreements
• Code of Conduct
• Doc Copyright

Tesla Optimus

• Dongfeng Motor

Dongfeng Motor will utilize advanced humanoid robots to help build its vehicles in China

A subsidiary of Dongfeng Motor Corporation has signed a strategic agreement with Ubtech Robotics, a leading smart service and humanoid robot company. As a result, Dongfeng will implement Ubtech’s humanoid robots into its vehicle manufacturing processes, joining a new era of next-generation mobility being assembled by next-generation robotic technology.

Dongfeng Motor Corporation has operated since the late 1960s and is the smallest of the “big four” Chinese state-owned automakers behind SAIC , Changan Auto, and FAW Group.

Since the Chinese government supports Dongfeng’s products, we don’t hear much about them here in the States, but we have covered the OEM briefly in the past, including its Hummer EV knockoff that debuted in 2022.

Dongfeng Liuzhou Motor Co., Ltd is a subsidiary of Dongfeng Motor Corporation that has been building vehicles just as long as its parent company, most recently crossovers and MPVs under the Fengxing nameplate, as well as vans and other vehicles for Lingzhi and Jingyi.

To aid in producing those Chinese vehicles, Dongfeng Liuzhou Motor has enlisted the help of Ubtech Robotics – a company founded in 2012 that currently offers a wide array of futuristic robots that can clean, deliver items, offer fitness training, and even companionship .

Dongfeng’s focus, however, is on one of Ubtech’s most advanced models – a humanoid robot called the Walker S, which is already helping build EVs for Chinese automakers like NIO .

Dongfeng to deploy Ubtech’s humanoid robots in China

Per a report by Car News China , Dongfeng Liuzhou Motor and Ubtech Robotics have signed a strategic cooperation agreement to deploy the latter’s industrial-grade humanoid robots across the former’s assembly lines in China.

The Walker S robots will be set with manufacturing tasks such as safety belt inspection, door lock tests, body quality checks, oil filling, and label applications. The report states that the humanoid robots will work in tandem with traditional automated manufacturing equipment to handle complex scenarios without human interference.

The Ubtech Walker S robots are 1.7 meters (5.57 feet) tall and equipped with a “fusion control” algorithm that enables real-time communication with factory systems and data sharing. The robots operate via 41 high-performance servo joints alongside advanced sensory capabilities that can perceive their environment and interact with objects and people.

Through its new partnership, Dongfeng Motor looks to streamline its production practices using cutting-edge robot technology while simultaneously promoting its capabilities as the way of the future in automotive production.

We’ve seen other automakers like Tesla introduce its own humanoid robot called Optimus, hinting at an impending future we hope won’t go the route of the T-1000s in the Terminator series. So far, we’re safe – most robots being used today are complex but offer simple tasks such as entertaining children, mowing your lawn, or delivering food and groceries.

The Tesla Optimus and Ubtech’s Walker S, however, are some of the most advanced humanoid robots we’ve seen alongside everything Boston Robotics has going on. This technology could serve several helpful use cases in delivering around-the-clock EV production without human error. What a time to be alive.

Check out the Walker S bot in action in the video below:

FTC: We use income earning auto affiliate links. More.

Scooter Doll is a writer, designer and tech enthusiast born in Chicago and based on the West Coast. When he’s not offering the latest tech how tos or insights, he’s probably watching Chicago sports. Please send any tips or suggestions, or dog photos to him at [email protected]

Manage push notifications

After Cruise Ban, What’s Next for Autonomous Vehicles?

Fully autonomous vehicles encountered another roadblock after Cruise’s ban in San Francisco. Here’s what that could mean for the future of AVs and a potential path forward. 

The future of autonomous vehicles is at a crossroads. On Oct. 4,  Cruise, a robotaxi company that uses the electric Chevy Bolt outfitted with autonomous gear to transport passengers across San Francisco, was told to halt operations in the city immediately. This happened after a woman crossing a downtown street was struck by a hit-and-run driver and then pinned beneath one of Cruise’s autonomous vehicles. 

California Department of Motor Vehicles suspended Cruise’s permit to operate stating that its vehicles weren’t safe for public operation and that the company had misrepresented safety information . Days after the California ruling, Cruise announced its fleets in other U.S. cities would now be operated with human oversight.

This follows a troubling trend regarding autonomous vehicles. 

2 Types of Autonomous Vehicles

• Driver-assisted: These vehicles deploy autonomous vehicle technology to support the driver, using features like backup cameras, adaptive cruise control, lane-assist and assisted driving. 
• Fully autonomous: These are driverless vehicles that don’t use steering wheels or braking and acceleration pedals and are controlled solely by autonomous vehicle technology. 

But for every autonomous vehicle death, there are thousands more caused by our driving behaviors. The reality is that by changing our mindset about the development and oversight of driverless cars, we can dramatically reduce the overall number of deaths caused by traffic accidents.

What’s Preventing Autonomous Vehicle Growth?

We see the world through the lenses of our experiences. Usually, we get to where we’re going without accidents or close calls, even when we engage in risky behaviors like speeding or texting. Yes, it’s risky, but it’s the devil we know, and we’ve grown dependent on its benefits. Driving falls within our societal risk tolerance.

Statistics tell a different story. The National Highway Traffic Safety Administration estimated that 42,795 people died in motor vehicle traffic crashes in 2022, according to its latest report . When it comes to driving safely, we simply aren’t that reliable.

Conversely, we’re often disproportionately frightened by crashes involving autonomous vehicles. We fear the unknown, and don’t understand the algorithms and decision-making capabilities of these “black boxes.”

Yet fully autonomous vehicles, or those without a steering wheel or braking and acceleration pedals, have the potential to save thousands of lives. With full implementation of driverless vehicles, some researchers estimate deaths could ultimately fall to 1 percent of current rates.

However, lawsuits and regulatory restrictions, such as the California DMV’s San Francisco prohibition, threaten to curtail these benefits by causing manufacturers to slow down driverless development in favor of driver-assisted technology. That puts responsibility on humans, not automation, which can be proven to cause far more deaths than driverless vehicles.

More on AI What Is Autonomous Trucking?

Why Driver-Assisted Vehicles Aren’t the Answer

The allure of driver-assisted technology is understandable. Driver-assist provides the gentle nudges we can appreciate, without giving up our sense of control. Few of us like to parallel park, so we gladly surrender that task. 

But human-assisted AV poses dynamic challenges, often causing the driver to over- or under-react. Under-reaction can happen through complacency; the driver trusting the autopilot “knows what it’s doing” and so the human doesn’t intervene until it’s too late, such as a malfunctioning adaptive cruise control.

An example of overreaction is when the driver is surprised by a driver-assist nudge and tries to bring the vehicle back to its intended course. For example, a driver who wants to change lanes but fails to signal. As the driver steers into the adjacent lane, the automation gently beeps or vibrates and directs the vehicle back in the original lane. The driver, not being familiar with the technology, turns the wheel even more aggressively until it overrides the automation feature. But depending on what level of override is allowed by the software, a human-automation tug-of-war may result. This can be physical, or cognitive, leading to a “What’s it doing now?” phenomenon. 

Driver-assisted autonomous vehicles aren’t a completely new phenomenon. It has parallels to the rise of autopilot in airplanes. The key to this evolution was the proficiency-based training all pilots are required to take and two reporting programs. Without fail, pilots must pass annual exams to demonstrate not only knowledge, skills, and abilities, but also to show proficiency in using the automation under different flight situations.

Meanwhile, the Aviation Safety Action Program and the Flight Operations Quality Assurance also collect both crew-informed and digital data to monitor trends and correct errors. Combined, those programs have amassed petabytes of data illuminating risk — both technological and human — in everyday flight operations, enabling manufacturers, airlines, and regulators to improve public safety through collaboration. Today, autopilot is treated as the norm, not an exception.

But the parallel between aviation and the roadways shows a distinct difference: U.S. drivers are not trained to proficiency. The last time you demonstrated your ability to drive a vehicle was likely as a teenager during your initial driving test at the DMV. 

This lack of proficiency, especially since driver-assisted technology has not been standardized among different manufacturers, makes the driver-automation coupling very risky. For human-assisted vehicles to work, drivers would need to be trained and a system for tracking data and reporting errors would need to be put in place. For this reason, the sooner the switch to full autonomy, the more lives can be saved.

More on AI A Feast of Sensor Data: Feeding Self-Driving Algorithms

Path Forward for a Safer Autonomous Vehicle Future

As the recent suspension of driverless vehicles demonstrates, government oversight of AVs hasn’t kept pace with advancing technology. Like all industries reliant on emerging technologies, most of the expertise lies in the private sector , where financial incentives compete for engineering skills. 

But when regulators overreact, it disincentivizes manufacturers to self-report the full set of data that could otherwise be used to guide technology development. Government overreaction has a chilling effect on manufacturer transparency.

In addition to the challenge of improving regulatory oversight, there are legal roadblocks to AV technology development. The U.S. tort system is not designed to manage risk. Its purpose is to compensate people who have already been harmed by “shifting the cost of harm to another person or entity who has erred in some legally cognizable way.” 

This approach means manufacturers will simply shift the technology from driverless to driver-assisted operations to shield themselves from liability. 

Instead, we need to see and understand the emerging risks of autonomous vehicles and manage them accordingly. Regulators such as the U.S. Department of Transportation, the National Highway Traffic Safety Administration and your local DMV should transition to risk-based regulation strategies, similar to the way the FAA embraced Safety Management Systems in the 1990s. To do this, regulators must partner with industry and labor associations. 

An alternative to the U.S. tort injury compensation model has been proposed , modeled after government action to incentivize vaccine development. In 1986, Congress created the Vaccine Injury Compensation Program in response to lawsuits against vaccine manufacturers when people suffered harm. The program compensates people injured by certain vaccines while limiting manufacturer liability, which stimulates vaccine development. 

The rationale: The private sector would not pursue vaccine research and development if the liabilities outweighed the potential returns on investment.

The clear, collaborative path forward is a national alternative death and injury compensation program, combined with collaborative public-private partnerships in the auto manufacturing and rideshare industries. 

Much like in other industries, these actions would accelerate research and development and make U.S. roads far safer. If autonomous vehicles can drastically reduce the number of deaths and injuries on our roads, failing to embrace these measures would leave more Americans at risk of dying at the hands of human drivers. 

While driver-assist technology provides short-term, incremental benefits, delaying full autonomy will cost thousands, if not tens of thousands, of lives. We have a once-in-a-generation opportunity to save them.

Recent Expert Contributors Articles